Skip to content

Implementation / Field notes

Measure LLM cost per completed workflow

Calculate complete-workflow cost with failed attempts and retries included, and report completion criteria and unmeasured cost categories.

By DriftRail · Updated

Measure LLM cost per completed workflow by dividing the relevant cost of all attempted workflows by the number that meet a defined completion criterion. Keep failed attempts in the numerator rather than reporting only the cost of the successful final call.

Define completion before calculating cost

A parsed answer, a reviewed correct answer, and a completed external action are different denominators. Choose the result the product needs and state it. Record partial and canceled tasks so a change in completion rules cannot masquerade as a cost improvement.

Use a complete attempt ledger

The fixture has three workflows. One succeeds immediately, one succeeds after a failed attempt, and one fails entirely. All four attempts consume synthetic cost units. Total cost is eight units and two workflows complete, producing four units per completed workflow. Excluding the failed workflow understates the effort needed to deliver those completions.

Build the real cost view

  1. Group provider attempts by task and separate duplicate telemetry deliveries.
  2. Include relevant model usage, tools, retrieval, retries, evaluation, and human review costs when the decision requires them.
  3. Keep estimates and invoice-reconciled values distinct, with currency and price-version context.
  4. Compare equivalent workloads and show quality and completion outcomes beside the cost.

Do not overclaim the accounting boundary

The example counts only its explicit synthetic units. It is not a price quote and does not estimate real provider billing, labor, or infrastructure. Unknown usage should remain unknown or estimated with a stated method; a missing usage record is not automatically free.

Run the worked example

Save the synthetic Node.js example and run node measure-llm-cost-per-workflow.mjs with Node.js 22 or newer. The assertions check the illustrated records and calculations; they do not test a deployed provider, SDK, or access control. No credentials or network connection are needed.

// Synthetic teaching fixture. No credentials, network calls, or production verification.
import assert from 'node:assert/strict';

const attempts = [
  { task: 'a', costUnits: 1, completed: true },
  { task: 'b', costUnits: 2, completed: false },
  { task: 'b', costUnits: 1, completed: true },
  { task: 'c', costUnits: 4, completed: false },
];
const totalCost = attempts.reduce((sum, a) => sum + a.costUnits, 0);
const completed = new Set(attempts.filter(a => a.completed).map(a => a.task)).size;
assert.equal(totalCost, 8);
assert.equal(completed, 2);
assert.equal(totalCost / completed, 4);
console.log(JSON.stringify({ totalSyntheticCost: totalCost, completedWorkflows: completed, costPerCompletion: totalCost / completed }));

References and further reading

Maintained by DriftRail. AI assisted the October 2026 content and template revision. Examples are illustrative; they are not measured product benchmarks. Suggest a correction.

From understanding to evidence

Observe your AI in production.

Explore event logging, detections, and review workflows in DriftRail.

Open DriftRail ↗Read the documentation →