Costing a job before you run it

Measure twenty, multiply by the rest

The reliable method has three steps and takes fifteen minutes. It is worth doing before anything that runs more than a few hundred times, and it converts an anxious guess into a number.

1. Run twenty real items. Not made-up ones: real ones, including a long one and a short one. Record the token usage for each.

tot_in = tot_out = 0
for item in sample_of_20:
    r = client.chat.completions.create(model=M, messages=build(item))
    tot_in  += r.usage.prompt_tokens
    tot_out += r.usage.completion_tokens
print(tot_in/20, tot_out/20)      # average per item

2. Multiply.

per item:   1,850 in, 240 out
50,000 items:
    input   92,500,000 tokens
    output  12,000,000 tokens
at $0.15 / $0.60 per million:
    $13.88 + $7.20 = $21.08
at $3.00 / $15.00 per million:
    $277.50 + $180.00 = $457.50

3. Add a third. Retries after failures, items that need a second pass, the run you abandon halfway and restart. A third is a realistic allowance; anything under a fifth is optimism.

Where the estimate goes wrong

The long tail. Twenty random items rarely include the outlier. If some of your documents are ten times longer than the average, and you have a thousand of them, they dominate the bill. Look at the distribution of your input sizes, not the mean, and if it has a tail, sample from the tail deliberately.

Output length is not what you asked for. “Be brief” is not a limit. If the average output in your sample is 900 tokens when you expected 200, either the instruction is being ignored or your idea of brief is not the model’s. Set max_tokens and see how often it truncates; that tells you the real distribution.

The conversation you forgot is resent. If the job is conversational rather than one-shot, every turn resends everything before it. A ten-turn conversation does not cost ten times a single turn: it costs closer to fifty times, because the input grows each turn.

Failures cost money. A request that returns a bad answer was still billed. A retry loop with three attempts triples the worst cases.

The comparison worth running

Do the twenty-item test on two model sizes, and score the outputs. This is the highest-value fifteen minutes in this whole module, because it converts “the big model is probably safer” into a number:

small model:  $21 for the job,  17/20 outputs acceptable
large model: $458 for the job,  19/20 outputs acceptable

$437 for three items out of twenty
->  or:  small model + hand-check the 3 flagged as uncertain

Very often the honest conclusion is that the small model plus a review step is better and cheaper than the large model unreviewed, because the large model’s failures are the ones nobody catches.

When the job is a subscription

The same arithmetic works in reverse. If you are on a $20 plan, ask what it would have cost through an API. Take a week of your actual conversations, estimate the tokens (most front-ends will show you, and a rough count of words divided by 0.75 is close enough) and price it.

People who do this split into two groups. Light chat users find their real usage would cost $2 to $4 a month, and the subscription is buying convenience and features rather than compute. Heavy users of long documents and thinking modes find they would spend far more than $20, and the plan is a bargain. Both are useful things to know, and neither is guessable from the outside.

Estimating without any code

If you are not going to write a script, the same method works by hand. Take five real items, run them in a chat front-end that shows token counts (most of the free ones do) and note the numbers. Where nothing shows counts, divide the character count of your input by four and the words of the output by 0.75.

That estimate will be out by perhaps 20%, which is far better than the order-of-magnitude error you would make by guessing. Add the third for retries as before, and you have a number good enough to decide with. Precision is not the point; knowing whether the answer is five dollars or five hundred is.

The number that ends most arguments

Before a meeting about whether to use AI for a task, work out the per-item cost and write it on one line: ”12 paise per invoice”, ”$0.004 per ticket”, ”$1.20 per contract review”. Almost every disagreement about whether this is worth doing dissolves once that number sits next to the cost of doing it the current way.