{"course":{"slug":"choosing-your-tools","title":"Choosing and Using the Tools","url":"https://addaly.com/learn/choosing-your-tools","level":"beginner","track":"work"},"lesson":{"slug":"estimating-a-job","title":"Costing a job before you run it","url":"https://addaly.com/learn/choosing-your-tools/estimating-a-job","position":32,"of":76,"minutes":8,"answer":"Run twenty real items, record actual token usage, multiply and add a third for retries.","keyIdea":"Run twenty real items, record actual token usage, multiply and add a third for retries, and run the same test on a small and a large model so the price of the extra accuracy is a number rather than an assumption."},"body":{"format":"markdown","text":"## Measure twenty, multiply by the rest\n\nThe reliable method has three steps and takes fifteen minutes. It is worth doing before anything that runs more than a few hundred times, and it converts an anxious guess into a number.\n\n**1. Run twenty real items.** Not made-up ones: real ones, including a long one and a short one. Record the token usage for each.\n\n```python\ntot_in = tot_out = 0\nfor item in sample_of_20:\n    r = client.chat.completions.create(model=M, messages=build(item))\n    tot_in  += r.usage.prompt_tokens\n    tot_out += r.usage.completion_tokens\nprint(tot_in/20, tot_out/20)      # average per item\n```\n\n**2. Multiply.**\n\n```\nper item:   1,850 in, 240 out\n50,000 items:\n    input   92,500,000 tokens\n    output  12,000,000 tokens\nat $0.15 / $0.60 per million:\n    $13.88 + $7.20 = $21.08\nat $3.00 / $15.00 per million:\n    $277.50 + $180.00 = $457.50\n```\n\n**3. Add a third.** Retries after failures, items that need a second pass, the run you abandon halfway and restart. A third is a realistic allowance; anything under a fifth is optimism.\n\n## Where the estimate goes wrong\n\n**The long tail.** Twenty random items rarely include the outlier. If some of your documents are ten times longer than the average, and you have a thousand of them, they dominate the bill. Look at the *distribution* of your input sizes, not the mean, and if it has a tail, sample from the tail deliberately.\n\n**Output length is not what you asked for.** \"Be brief\" is not a limit. If the average output in your sample is 900 tokens when you expected 200, either the instruction is being ignored or your idea of brief is not the model's. Set `max_tokens` and see how often it truncates; that tells you the real distribution.\n\n**The conversation you forgot is resent.** If the job is conversational rather than one-shot, every turn resends everything before it. A ten-turn conversation does not cost ten times a single turn: it costs closer to fifty times, because the input grows each turn.\n\n> Figure: Costing a job in fifteen minutes. Steps: 1. Run twenty real items (Real ones, including one long and one short, and record the token usage the response reports for each.); 2. Take the average (In this example, 1,850 tokens in and 240 out per item.); 3. Multiply by the whole job (Fifty thousand items: 92.5 million tokens in, 12 million out.); 4. Add a third (Retries, second passes, the run you abandon halfway. Anything under a fifth is optimism.); 5. Run the same twenty on a smaller model (Score both. Now the price of the extra accuracy is a number rather than an assumption.). Precision is not the point. Knowing whether the answer is twenty dollars or five hundred is, and the per-item figure (twelve paise an invoice, a fifth of a cent a ticket) is the number that ends most arguments about whether to do this at all.\n\n**Failures cost money.** A request that returns a bad answer was still billed. A retry loop with three attempts triples the worst cases.\n\n## The comparison worth running\n\nDo the twenty-item test on two model sizes, and score the outputs. This is the highest-value fifteen minutes in this whole module, because it converts \"the big model is probably safer\" into a number:\n\n```\nsmall model:  $21 for the job,  17/20 outputs acceptable\nlarge model: $458 for the job,  19/20 outputs acceptable\n\n$437 for three items out of twenty\n->  or:  small model + hand-check the 3 flagged as uncertain\n```\n\nVery often the honest conclusion is that the small model plus a review step is better and cheaper than the large model unreviewed, because the large model's failures are the ones nobody catches.\n\n## When the job is a subscription\n\nThe same arithmetic works in reverse. If you are on a $20 plan, ask what it would have cost through an API. Take a week of your actual conversations, estimate the tokens (most front-ends will show you, and a rough count of words divided by 0.75 is close enough) and price it.\n\nPeople who do this split into two groups. Light chat users find their real usage would cost $2 to $4 a month, and the subscription is buying convenience and features rather than compute. Heavy users of long documents and thinking modes find they would spend far more than $20, and the plan is a bargain. Both are useful things to know, and neither is guessable from the outside.\n\n## Estimating without any code\n\nIf you are not going to write a script, the same method works by hand. Take five real items, run them in a chat front-end that shows token counts (most of the free ones do) and note the numbers. Where nothing shows counts, divide the character count of your input by four and the words of the output by 0.75.\n\nThat estimate will be out by perhaps 20%, which is far better than the order-of-magnitude error you would make by guessing. Add the third for retries as before, and you have a number good enough to decide with. Precision is not the point; knowing whether the answer is five dollars or five hundred is.\n\n## The number that ends most arguments\n\nBefore a meeting about whether to use AI for a task, work out the per-item cost and write it on one line: *\"12 paise per invoice\"*, *\"$0.004 per ticket\"*, *\"$1.20 per contract review\"*. Almost every disagreement about whether this is worth doing dissolves once that number sits next to the cost of doing it the current way."},"check":{"question":"A twenty-item pilot predicts $21 for a 50,000-item job, but the real run costs $190. The prompt and model were unchanged. What is the most likely cause?","options":[{"text":"The provider applied surge pricing during a period of high demand across the platform.","correct":false,"tempting":"Invokes demand pricing that these APIs do not use; published per-token rates do not vary with load."},{"text":"Token prices rose between the pilot and the run, which is common given how quickly this field moves.","correct":false,"tempting":"Points at a trend that has generally moved the other way, and a nine-fold jump is far beyond any price revision."},{"text":"The sample of twenty missed the long tail, and a minority of much larger inputs dominated the total.","correct":true},{"text":"The model produced longer outputs at scale because later items in a batch receive less attention per item.","correct":false,"tempting":"Invents a batch-position effect; each request is independent and knows nothing about its place in a queue."}]},"prev":{"slug":"tokens-as-the-unit","title":"The unit you are billed in","url":"https://addaly.com/learn/choosing-your-tools/tokens-as-the-unit"},"next":{"slug":"caching-and-batching","title":"The two discounts almost nobody uses","url":"https://addaly.com/learn/choosing-your-tools/caching-and-batching"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Costing a job before you run it', in Choosing and Using the Tools. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/choosing-your-tools/estimating-a-job (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Costing a job before you run it","courseTitle":"Choosing and Using the Tools","module":{"index":4,"slug":"what-it-costs","title":"What it actually costs"},"answer":null,"keyIdea":"Run twenty real items, record actual token usage, multiply and add a third for retries, and run the same test on a small and a large model so the price of the extra accuracy is a number rather than an assumption.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"flow","steps":[{"label":"Run twenty real items","detail":"Real ones, including one long and one short, and record the token usage the response reports for each."},{"label":"Take the average","detail":"In this example, 1,850 tokens in and 240 out per item."},{"label":"Multiply by the whole job","detail":"Fifty thousand items: 92.5 million tokens in, 12 million out."},{"label":"Add a third","detail":"Retries, second passes, the run you abandon halfway. Anything under a fifth is optimism."},{"label":"Run the same twenty on a smaller model","detail":"Score both. Now the price of the extra accuracy is a number rather than an assumption."}],"title":"Costing a job in fifteen minutes","caption":"Precision is not the point. Knowing whether the answer is twenty dollars or five hundred is, and the per-item figure (twelve paise an invoice, a fifth of a cent a ticket) is the number that ends most arguments about whether to do this at all."},"alt":"Figure: Costing a job in fifteen minutes. Steps: 1. Run twenty real items (Real ones, including one long and one short, and record the token usage the response reports for each.); 2. Take the average (In this example, 1,850 tokens in and 240 out per item.); 3. Multiply by the whole job (Fifty thousand items: 92.5 million tokens in, 12 million out.); 4. Add a third (Retries, second passes, the run you abandon halfway. Anything under a fifth is optimism.); 5. Run the same twenty on a smaller model (Score both. Now the price of the extra accuracy is a number rather than an assumption.). Precision is not the point. Knowing whether the answer is twenty dollars or five hundred is, and the per-item figure (twelve paise an invoice, a fifth of a cent a ticket) is the number that ends most arguments about whether to do this at all."}},"blocks":[{"t":"h2","id":"measure-twenty-multiply-by-the-rest","runs":[{"text":"Measure twenty, multiply by the rest"}]},{"t":"p","runs":[{"text":"The reliable method has three steps and takes fifteen minutes. It is worth doing before anything that runs more than a few hundred times, and it converts an anxious guess into a number."}]},{"t":"p","runs":[{"text":"1. Run twenty real items.","b":true},{"text":" Not made-up ones: real ones, including a long one and a short one. Record the token usage for each."}]},{"t":"code","lang":"python","label":"Python","text":"tot_in = tot_out = 0\nfor item in sample_of_20:\n    r = client.chat.completions.create(model=M, messages=build(item))\n    tot_in  += r.usage.prompt_tokens\n    tot_out += r.usage.completion_tokens\nprint(tot_in/20, tot_out/20)      # average per item"},{"t":"p","runs":[{"text":"2. Multiply.","b":true}]},{"t":"code","lang":"","label":"Text","text":"per item:   1,850 in, 240 out\n50,000 items:\n    input   92,500,000 tokens\n    output  12,000,000 tokens\nat $0.15 / $0.60 per million:\n    $13.88 + $7.20 = $21.08\nat $3.00 / $15.00 per million:\n    $277.50 + $180.00 = $457.50"},{"t":"p","runs":[{"text":"3. Add a third.","b":true},{"text":" Retries after failures, items that need a second pass, the run you abandon halfway and restart. A third is a realistic allowance; anything under a fifth is optimism."}]},{"t":"h2","id":"where-the-estimate-goes-wrong","runs":[{"text":"Where the estimate goes wrong"}]},{"t":"p","runs":[{"text":"The long tail.","b":true},{"text":" Twenty random items rarely include the outlier. If some of your documents are ten times longer than the average, and you have a thousand of them, they dominate the bill. Look at the "},{"text":"distribution","i":true},{"text":" of your input sizes, not the mean, and if it has a tail, sample from the tail deliberately."}]},{"t":"p","runs":[{"text":"Output length is not what you asked for.","b":true},{"text":" “Be brief” is not a limit. If the average output in your sample is 900 tokens when you expected 200, either the instruction is being ignored or your idea of brief is not the model’s. Set "},{"text":"max_tokens","code":true},{"text":" and see how often it truncates; that tells you the real distribution."}]},{"t":"p","runs":[{"text":"The conversation you forgot is resent.","b":true},{"text":" If the job is conversational rather than one-shot, every turn resends everything before it. A ten-turn conversation does not cost ten times a single turn: it costs closer to fifty times, because the input grows each turn."}]},{"t":"p","runs":[{"text":"Failures cost money.","b":true},{"text":" A request that returns a bad answer was still billed. A retry loop with three attempts triples the worst cases."}]},{"t":"h2","id":"the-comparison-worth-running","runs":[{"text":"The comparison worth running"}]},{"t":"p","runs":[{"text":"Do the twenty-item test on two model sizes, and score the outputs. This is the highest-value fifteen minutes in this whole module, because it converts “the big model is probably safer” into a number:"}]},{"t":"code","lang":"","label":"Text","text":"small model:  $21 for the job,  17/20 outputs acceptable\nlarge model: $458 for the job,  19/20 outputs acceptable\n\n$437 for three items out of twenty\n->  or:  small model + hand-check the 3 flagged as uncertain"},{"t":"p","runs":[{"text":"Very often the honest conclusion is that the small model plus a review step is better and cheaper than the large model unreviewed, because the large model’s failures are the ones nobody catches."}]},{"t":"h2","id":"when-the-job-is-a-subscription","runs":[{"text":"When the job is a subscription"}]},{"t":"p","runs":[{"text":"The same arithmetic works in reverse. If you are on a $20 plan, ask what it would have cost through an API. Take a week of your actual conversations, estimate the tokens (most front-ends will show you, and a rough count of words divided by 0.75 is close enough) and price it."}]},{"t":"p","runs":[{"text":"People who do this split into two groups. Light chat users find their real usage would cost $2 to $4 a month, and the subscription is buying convenience and features rather than compute. Heavy users of long documents and thinking modes find they would spend far more than $20, and the plan is a bargain. Both are useful things to know, and neither is guessable from the outside."}]},{"t":"h2","id":"estimating-without-any-code","runs":[{"text":"Estimating without any code"}]},{"t":"p","runs":[{"text":"If you are not going to write a script, the same method works by hand. Take five real items, run them in a chat front-end that shows token counts (most of the free ones do) and note the numbers. Where nothing shows counts, divide the character count of your input by four and the words of the output by 0.75."}]},{"t":"p","runs":[{"text":"That estimate will be out by perhaps 20%, which is far better than the order-of-magnitude error you would make by guessing. Add the third for retries as before, and you have a number good enough to decide with. Precision is not the point; knowing whether the answer is five dollars or five hundred is."}]},{"t":"h2","id":"the-number-that-ends-most-arguments","runs":[{"text":"The number that ends most arguments"}]},{"t":"p","runs":[{"text":"Before a meeting about whether to use AI for a task, work out the per-item cost and write it on one line: "},{"text":"”12 paise per invoice”","i":true},{"text":", "},{"text":"”$0.004 per ticket”","i":true},{"text":", "},{"text":"”$1.20 per contract review”","i":true},{"text":". Almost every disagreement about whether this is worth doing dissolves once that number sits next to the cost of doing it the current way."}]}],"check":{"question":[{"text":"A twenty-item pilot predicts $21 for a 50,000-item job, but the real run costs $190. The prompt and model were unchanged. What is the most likely cause?"}],"options":[{"text":[{"text":"The provider applied surge pricing during a period of high demand across the platform."}],"correct":false,"tempting":[{"text":"Invokes demand pricing that these APIs do not use; published per-token rates do not vary with load."}]},{"text":[{"text":"Token prices rose between the pilot and the run, which is common given how quickly this field moves."}],"correct":false,"tempting":[{"text":"Points at a trend that has generally moved the other way, and a nine-fold jump is far beyond any price revision."}]},{"text":[{"text":"The sample of twenty missed the long tail, and a minority of much larger inputs dominated the total."}],"correct":true,"tempting":null},{"text":[{"text":"The model produced longer outputs at scale because later items in a batch receive less attention per item."}],"correct":false,"tempting":[{"text":"Invents a batch-position effect; each request is independent and knows nothing about its place in a queue."}]}],"order":[0,1,2,3],"why":null},"prev":{"slug":"tokens-as-the-unit","title":"The unit you are billed in","minutes":8},"next":{"slug":"caching-and-batching","title":"The two discounts almost nobody uses","minutes":8}}}