Token Costs Don't Measure AI Capacity: Build a Capacity Ledger Instead

Six · 10 min read · June 16, 2026

Token Costs Don't Measure AI Capacity: Build a Capacity Ledger Instead

Say you lead product marketing, and your team has started using AI agents to write a weekly brief on each of your main competitors: what they changed in pricing, what they launched, how they describe themselves. At the end of the month the finance lead sends over the AI bill. It doubled. The dashboard says the agents finished twenty briefs this month, up from ten.

So did your capacity double?

Neither number can tell you. The bill says what the models consumed, and the task count says how often a run reached an end. What you need to know is how many of those briefs your team trusted enough to use in a pricing or packaging decision, and how many hours your marketers spent checking sources first.

It can fool you the other way, too. If next month's token cost falls by half, maybe the work got more efficient. Or maybe the briefs got shorter, the source checking moved into someone's evening, and the hard calls quietly went back to people.

How should you measure AI agent capacity?

Count the work your team accepts and uses. Here's the definition we work from: usable AI capacity is how many briefs your team accepted and used this month, what they cost and how long people spent checking them, within the budget and staff you already have. The token bill is one line in what they cost. A Capacity Ledger connects that spend to the work it made usable.

What tokens are good for

Tokens do measure something real. MLCommons uses tokens per second as the throughput metric in its Llama 2 70B inference benchmark, because queries "can have different input and output token lengths." That tells you how fast a model is served under test conditions, not how many usable briefs a thousand tokens bought.

Take the competitor briefs. A tight brief that names exactly which pricing tier a rival changed can take more model effort than a generic summary of the rival's website, and it's the one your team can act on. Skimp on sources and the cost moves to a person, who spends an hour opening pricing pages before anyone will trust the brief. And sometimes the expensive run is the bargain, because it turns up sources the team reuses for the next five decisions.

Finished tasks aren't capacity either

One brief might involve several model calls, a few retries and a review, while another finishes in seconds and fails review. So count a unit of capacity only when the work reaches an outcome you defined before the period started.

This is where volume gets seductive. Suppose the agents turn out twelve briefs one week instead of the usual six, and the dashboard looks busy. But if eight of them are waiting for review, three cite an old pricing page and none reaches the person deciding on packaging, the team has built up inventory.

DORA's guidance on work-in-process limits describes the same trap in software teams: managers hand people several tasks at once to raise throughput, and "tasks take longer to get done, and the team burns out in the process." Its advice is to prioritize, limit how much is in progress, make the work visible and measure lead time through the whole system. Every brief you start before the last one is reviewed adds to the wait for all of them.

Build the Capacity Ledger in six columns

A Capacity Ledger is a weekly or monthly record for one kind of AI work. Keep unlike work in separate rows, because "tasks per agent" means nothing when one task is a two-line call summary and another is a code change.

What counts as one unit Flow Outcome Money People Constraint and decision
The unit of work and its acceptance standard Admitted, average in progress, finished, time to acceptance Accepted, revised, rejected, used Actual cost where you have it, labeled estimates where you don't, plus the token usage behind it Review time, approvals, escalations, manual repair The main limit this period, and whether to repeat, broaden, narrow, pause or redesign

1. Decide what counts as one unit

A unit is a finished result with a named person who receives it and a standard it has to meet. Write it down before the period begins, or the definition will drift toward whatever makes the dashboard look good. A simple template works:

One [piece of work] accepted by [owner] for [decision or use] against [standard].

For the briefs, that might read:

One competitor brief accepted by product marketing for a packaging decision, with current primary sources linked and unsupported claims removed.

Now when someone says the agents finished a brief, you can open it and check: are the sources current and primary, and did someone strip out the claims it couldn't support?

2. Record how work moved, not only what finished

Track how many briefs you asked for, how many were in progress on average, how many finished and how long each took to reach acceptance. When the pile in progress grows, either more briefs are coming in or each one is waiting longer for review, context, access or a fix, and you need to know which.

John Little's 1961 proof ties the two together: the average number of items in a system equals the rate they arrive times the average time each spends there, L = λW. In a retrospective fifty years later, Little noted that the relation is exact after the fact, once you know what happened: "we are in the measurement business, not the forecasting business."

Use it that way for the briefs. If the number in progress keeps climbing while finished briefs per week stay flat, each brief is spending longer in the system, which means it's waiting somewhere. Find where before you admit more work.

3. Separate finished, accepted and used

Finished means the run reached an end. Accepted means someone responsible checked the brief against its standard and approved it. Used means it went into the packaging decision or sales note it was written for. These often come apart, so count each state and the reason:

Don't turn "used" into an invented revenue figure. Sometimes the honest record is that a decision moved forward, a risk came up early or the team now has a source list it trusts, and that's what you write down.

4. Put the money next to the outcome

For one competitor brief, write down what its runs were charged, from whichever bill or usage record shows it, and label anything you had to estimate. Keep that apart from wider costs like tool subscriptions, so when the finance lead asks where a number came from, you can point to the record.

The FinOps Foundation's terminology defines unit economics as "the revenues and costs associated with a defined and measurable unit of a product or service." The ledger borrows the defined unit and leaves the profit claim alone, because for most knowledge work you can see what an accepted brief cost long before you can say what it earned. The views worth keeping are:

A cheap week isn't a good week if nothing in it got used, so read cost per accepted brief next to the number used.

5. Count review time as part of the cost

Handing work to an agent still means writing the brief, judging the result, approving it and fixing what went wrong, so people's attention is part of AI capacity. Record review time, revision rounds, escalations and any substantial manual repair. You don't need to price every minute; the minutes and the count of interventions show where the work gets stuck. If model cost falls while your senior marketer spends twice as long checking sources, you haven't gained free capacity.

DORA found the same tension in software teams. In a March 2026 analysis, Jessica Baolin and Nathen Harvey write that "the time saved in creation is frequently re-allocated to auditing and verification," and cite DORA's 2025 finding that higher AI adoption "is associated with an increase in both software delivery throughput and software delivery instability." That's software research, but the briefs have the same shape: a fast first draft tells you little until you know how long the checking took.

6. Name what held you back, then decide

The binding constraint is whatever most limited accepted work this period: budget, review time, source quality, access to a system, a tool that kept failing, a vague brief, slow approvals or a lack of demand. End every row with a decision:

The constraint tells you where the next dollar or hour should go. If review is the limit, a bigger budget just makes the queue longer, and if the sources are wrong, a faster model gets you wrong briefs sooner.

What a Capacity Ledger looks like

The ledger should be small enough to go through in one meeting and specific enough to change what work you take on next. Say the same team also has an agent writing a weekly summary of support trends. The month the bill doubled might look like this:

Work and period Flow Outcome Money People Constraint and decision
Weekly competitor briefs 24 admitted; 5 in progress on average; 20 finished 16 accepted; 12 used; 3 revised; 4 rejected Actual and estimated cost reconciled; token usage linked 6 review hours; 3 escalations Stale sources: redesign the source check before broadening
Support trend summaries 4 admitted; 4 finished 4 accepted; 1 used in planning Cost steady against the previous period 25 review minutes Weak demand: pause for two weeks and confirm who it's for

On a usage dashboard both rows would look healthy. In the ledger, the first row doesn't justify more briefs yet: most of the rejections and escalations traced back to stale sources, so the source check gets fixed first. The second row looks efficient until you check what got used: every summary was accepted and only one made it into planning, so the next step is to ask who they're for.

How do you compare agents or models fairly?

Compare two agents or models only on the same briefs, with the same sources, standard and reviewers, and read the whole row for each, including how it fails. Before you crown the agent with the lowest token cost, check whether it was handed the easy competitors. The model with the most completions may be the one your reviewers have been quietly fixing.

What patterns should the ledger show you?

After a month or two of rows, the briefs will show you things a usage dashboard can't. Keep them as separate notes rather than one score, because each one changes what you do next:

Expect that last one. Fix the constraint you can see and the next one turns up somewhere else.

Start with one kind of work

Pick one recurring piece of work with a real recipient and a visible moment of acceptance, like the competitor brief. Look back over four to eight weeks if you have the records, mark the gaps honestly and pick one constraint to work on. A company-wide dashboard can wait until your team agrees on what one accepted brief is.

For the first ledger:

  1. Write the unit definition.
  2. Choose a weekly or monthly period.
  3. Count what was admitted, in progress, finished, accepted and used.
  4. Reconcile actual and estimated cost without merging records that measure different things.
  5. Estimate review time conservatively, or count interventions instead.
  6. Name the binding constraint.
  7. Decide whether to repeat, broaden, narrow, pause or redesign.

Then keep the definition steady long enough to learn something. If you change the brief, the model, the sources, the review standard or who approves the work, note it in the ledger, so nobody reads the shift that follows as a performance trend.

Measure what became possible

Go back to the bill that doubled. With a ledger, you can give the finance lead a real answer: this many briefs were accepted, this many were used, this is what each cost and this is how long your team spent reviewing them. If the number used rose with the bill, you bought capacity. If it didn't, the ledger shows you what got in the way.

Say the briefs ran on Supanova atoms. Finished work comes back for your review, and that's where each brief is accepted or not. Usage is shown down to fractions of a cent, so the money column can start from what you were charged. The Learning Signal shows how often your workforce gets it right the first time, this month versus last, across everything reviewed in your workspace. It won't split out the briefs, so their accepted count, whether each got used and next period's decision still go in their own row.

To start, open one brief that mattered last month. Write down what it cost, how long your team spent reviewing and fixing it, and whether anyone accepted it and used it. Then ask the question the ledger exists to answer: what useful work can this setup reliably make possible next?

Start at supanova.team

Sources

AI Agent CapacityWorkforce PlanningAI MeasurementAI OperationsAI Costs