{"course":{"slug":"context-engineering","title":"Context Engineering","url":"https://addaly.com/learn/context-engineering","level":"advanced","track":"build"},"lesson":{"slug":"what-a-token-costs","title":"The arithmetic of a request","url":"https://addaly.com/learn/context-engineering/what-a-token-costs","position":40,"of":87,"minutes":10,"answer":"Input, output, cached input and cache writes are four different prices, output is the expensive one, and a conversation bills quadratically because every turn resends the whole history, so measure cost per resolved outcome.","keyIdea":"Input, output, cached input and cache writes are four different prices, output is the expensive one, and a conversation bills quadratically because every turn resends the whole history, so measure cost per resolved outcome, not per call."},"body":{"format":"markdown","text":"## Four prices, not one\n\nEvery hosted model has at least four rates, and confusing them is how a cost model ends up wrong by a factor of five.\n\n- **Input**, per million tokens. The base rate.\n- **Output**, per million tokens. Typically four to eight times the input rate (five times on every current Anthropic model and on OpenAI's newest; six to eight times on several OpenAI and Google models, checked October 2026), because generation is sequential and cannot be batched the same way.\n- **Cached input**, when a prefix is reused. Often around a tenth of the base input rate.\n- **Cache write**, a premium on the first call that stores the prefix: commonly around 1.25 times base input.\n\nThere is usually a **batch** rate as well, around half price for work that can wait hours.\n\nExact numbers change every few months and differ by an order of magnitude between the cheapest small model and the largest frontier one, so look them up rather than memorising them. The ratios above are stable and are what the reasoning depends on.\n\n> Figure: Four rates on the same request. Bars (% of the base input rate): Cached input 10; Batch input 50; Input 100; Cache write 125; Output 400. Exact prices change every few months and differ by an order of magnitude between the smallest model and the largest, so look them up rather than memorising them. The ratios are stable, and they are what the arithmetic needs. Output is usually a few per cent of the tokens but, at several times the input rate, a much larger share of the bill (more than a tenth uncached, and up to a third or more when most input is cached), which is why asking for a shorter answer is a real cost lever.\n\nThe formula for one call:\n\n```\ncost = uncached_in × p_in\n     + cached_in   × p_in × 0.1\n     + written_in  × p_in × 1.25\n     + out         × p_out\n```\n\nEvery response tells you those four counts in its usage block. Log them. A cost model built from logged usage is right; one built from estimates drifts within a month.\n\n## The number that surprises everybody\n\nA conversation is billed quadratically.\n\nEach turn resends the entire history. Ten turns of 500 tokens each are not 5,000 tokens of billing. They are 500 + 1,000 + 1,500 + ... + 5,000 = **27,500 input tokens**, for 5,000 tokens of actual content. Twenty turns is 105,000. The content grew linearly; the bill grew with the square.\n\nThis is why a chat product's cost per user is dominated by long conversations rather than by many short ones, and why the average conversation length is one of the most important numbers in your cost model. It is also the strongest argument for prefix caching, which turns most of that resent history into cached reads at a tenth of the price, and for the compaction techniques in the next module.\n\nDo the arithmetic for your own product before you price it. A team assuming linear growth will underestimate a twenty-turn conversation by roughly a factor of ten.\n\n## Where the money actually is\n\nRun this for one real request and you will usually find the same shape:\n\n- Retrieved passages: the largest single block, often half the input.\n- History: second, and growing.\n- Tool definitions: a fixed tax, larger than teams expect.\n- System prompt and exemplars: smaller than everyone assumes.\n- Output: few tokens, high rate. At 3% of the tokens and a 5x price it is about 13% of an uncached bill (about 20% at 8x), and it climbs toward a third or more when most of the input is billed at cache-read rates.\n\nTwo consequences for where to spend an afternoon. Cutting output length is disproportionately valuable, because output is the expensive rate; asking for a shorter answer is a genuine cost lever, not a stylistic one. And cutting retrieved passages from ten to five, which the k-curve lesson may already have told you to do for quality reasons, is usually the largest single saving available.\n\n## Cost per unit of value, not cost per call\n\nCost per call is the wrong denominator for a decision. The right one is cost per resolved ticket, per correct extraction, per accepted draft.\n\nA change that halves the cost per call and raises the failure rate from 8% to 20% has increased the cost per resolution, because every failure is retried, escalated to a human, or abandoned. A human escalation typically costs orders of magnitude more than any inference call, so a small accuracy loss can swamp a large token saving.\n\nWrite it down explicitly:\n\n```\ncost_per_resolution = (calls × cost_per_call) / resolved\n```\n\nThis is the number that survives contact with a finance conversation, and it is the one that prevents the classic mistake of celebrating a 40% infrastructure saving that quietly doubled the support workload.\n\n## Estimating before you build\n\nA usable envelope, per feature:\n\n1. Requests per day.\n2. Input tokens per request, from the budget ledger in module one.\n3. Output tokens per request, at the 95th percentile rather than the mean.\n4. Expected cache hit rate: assume 0 until measured.\n\nMultiply, then double it. The doubling is not pessimism; it covers retries, evaluation runs, the internal team's usage, and the requests that turn out to be four calls rather than one because a tool was invoked.\n\nA forecast that comes in at twice the eventual bill costs you a conversation. One that comes in at half costs you the project's credibility in month two."},"check":{"question":"A ten-turn conversation with 500 tokens added per turn is billed for roughly 27,500 input tokens rather than 5,000. Why?","options":[{"text":"Every turn resends the full prior history, so the billed total is the sum of a growing prefix.","correct":true},{"text":"Output tokens are billed at a higher rate, and ten turns of output accumulate.","correct":false,"tempting":"Output is indeed dearer, but the figure quoted is input tokens, and the ratio described is about resending rather than rate."},{"text":"The chat template adds delimiters to every message, multiplying the count.","correct":false,"tempting":"Template overhead is real but is a few tokens per message, nowhere near a fivefold increase."},{"text":"Prefix caching writes each prefix at a premium, inflating the counted input.","correct":false,"tempting":"Cache writes carry a price premium, not a token multiplier, and caching reduces the effective cost of resent history."}]},"prev":{"slug":"attention-sinks-and-the-first-tokens","title":"Why the first tokens are special","url":"https://addaly.com/learn/context-engineering/attention-sinks-and-the-first-tokens"},"next":{"slug":"prefill-decode-and-where-the-time-goes","title":"Where the milliseconds go","url":"https://addaly.com/learn/context-engineering/prefill-decode-and-where-the-time-goes"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The arithmetic of a request', in Context Engineering. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/context-engineering/what-a-token-costs (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The arithmetic of a request","courseTitle":"Context Engineering","module":{"index":5,"slug":"cost-cache-and-latency","title":"Cost, caching and latency"},"answer":null,"keyIdea":"Input, output, cached input and cache writes are four different prices, output is the expensive one, and a conversation bills quadratically because every turn resends the whole history, so measure cost per resolved outcome, not per call.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure 5.1","figure":{"type":"bars","unit":"% of the base input rate","title":"Four rates on the same request","series":[{"label":"Cached input","value":10},{"label":"Batch input","value":50},{"label":"Input","value":100},{"label":"Cache write","value":125},{"label":"Output","value":400,"highlight":true}],"caption":"Exact prices change every few months and differ by an order of magnitude between the smallest model and the largest, so look them up rather than memorising them. The ratios are stable, and they are what the arithmetic needs. Output is usually a few per cent of the tokens but, at several times the input rate, a much larger share of the bill (more than a tenth uncached, and up to a third or more when most input is cached), which is why asking for a shorter answer is a real cost lever."},"alt":"Figure: Four rates on the same request. Bars (% of the base input rate): Cached input 10; Batch input 50; Input 100; Cache write 125; Output 400. Exact prices change every few months and differ by an order of magnitude between the smallest model and the largest, so look them up rather than memorising them. The ratios are stable, and they are what the arithmetic needs. Output is usually a few per cent of the tokens but, at several times the input rate, a much larger share of the bill (more than a tenth uncached, and up to a third or more when most input is cached), which is why asking for a shorter answer is a real cost lever."}},"blocks":[{"t":"h2","id":"four-prices-not-one","runs":[{"text":"Four prices, not one"}]},{"t":"p","runs":[{"text":"Every hosted model has at least four rates, and confusing them is how a cost model ends up wrong by a factor of five."}]},{"t":"ul","items":[[{"text":"Input","b":true},{"text":", per million tokens. The base rate."}],[{"text":"Output","b":true},{"text":", per million tokens. Typically four to eight times the input rate (five times on every current Anthropic model and on OpenAI’s newest; six to eight times on several OpenAI and Google models, checked October 2026), because generation is sequential and cannot be batched the same way."}],[{"text":"Cached input","b":true},{"text":", when a prefix is reused. Often around a tenth of the base input rate."}],[{"text":"Cache write","b":true},{"text":", a premium on the first call that stores the prefix: commonly around 1.25 times base input."}]]},{"t":"p","runs":[{"text":"There is usually a "},{"text":"batch","b":true},{"text":" rate as well, around half price for work that can wait hours."}]},{"t":"p","runs":[{"text":"Exact numbers change every few months and differ by an order of magnitude between the cheapest small model and the largest frontier one, so look them up rather than memorising them. The ratios above are stable and are what the reasoning depends on."}]},{"t":"p","runs":[{"text":"The formula for one call:"}]},{"t":"code","lang":"","label":"Text","text":"cost = uncached_in × p_in\n     + cached_in   × p_in × 0.1\n     + written_in  × p_in × 1.25\n     + out         × p_out"},{"t":"p","runs":[{"text":"Every response tells you those four counts in its usage block. Log them. A cost model built from logged usage is right; one built from estimates drifts within a month."}]},{"t":"h2","id":"the-number-that-surprises-everybody","runs":[{"text":"The number that surprises everybody"}]},{"t":"p","runs":[{"text":"A conversation is billed quadratically."}]},{"t":"p","runs":[{"text":"Each turn resends the entire history. Ten turns of 500 tokens each are not 5,000 tokens of billing. They are 500 + 1,000 + 1,500 + ... + 5,000 = "},{"text":"27,500 input tokens","b":true},{"text":", for 5,000 tokens of actual content. Twenty turns is 105,000. The content grew linearly; the bill grew with the square."}]},{"t":"p","runs":[{"text":"This is why a chat product’s cost per user is dominated by long conversations rather than by many short ones, and why the average conversation length is one of the most important numbers in your cost model. It is also the strongest argument for prefix caching, which turns most of that resent history into cached reads at a tenth of the price, and for the compaction techniques in the next module."}]},{"t":"p","runs":[{"text":"Do the arithmetic for your own product before you price it. A team assuming linear growth will underestimate a twenty-turn conversation by roughly a factor of ten."}]},{"t":"h2","id":"where-the-money-actually-is","runs":[{"text":"Where the money actually is"}]},{"t":"p","runs":[{"text":"Run this for one real request and you will usually find the same shape:"}]},{"t":"ul","items":[[{"text":"Retrieved passages: the largest single block, often half the input."}],[{"text":"History: second, and growing."}],[{"text":"Tool definitions: a fixed tax, larger than teams expect."}],[{"text":"System prompt and exemplars: smaller than everyone assumes."}],[{"text":"Output: few tokens, high rate. At 3% of the tokens and a 5x price it is about 13% of an uncached bill (about 20% at 8x), and it climbs toward a third or more when most of the input is billed at cache-read rates."}]]},{"t":"p","runs":[{"text":"Two consequences for where to spend an afternoon. Cutting output length is disproportionately valuable, because output is the expensive rate; asking for a shorter answer is a genuine cost lever, not a stylistic one. And cutting retrieved passages from ten to five, which the k-curve lesson may already have told you to do for quality reasons, is usually the largest single saving available."}]},{"t":"h2","id":"cost-per-unit-of-value-not-cost-per-call","runs":[{"text":"Cost per unit of value, not cost per call"}]},{"t":"p","runs":[{"text":"Cost per call is the wrong denominator for a decision. The right one is cost per resolved ticket, per correct extraction, per accepted draft."}]},{"t":"p","runs":[{"text":"A change that halves the cost per call and raises the failure rate from 8% to 20% has increased the cost per resolution, because every failure is retried, escalated to a human, or abandoned. A human escalation typically costs orders of magnitude more than any inference call, so a small accuracy loss can swamp a large token saving."}]},{"t":"p","runs":[{"text":"Write it down explicitly:"}]},{"t":"code","lang":"","label":"Text","text":"cost_per_resolution = (calls × cost_per_call) / resolved"},{"t":"p","runs":[{"text":"This is the number that survives contact with a finance conversation, and it is the one that prevents the classic mistake of celebrating a 40% infrastructure saving that quietly doubled the support workload."}]},{"t":"h2","id":"estimating-before-you-build","runs":[{"text":"Estimating before you build"}]},{"t":"p","runs":[{"text":"A usable envelope, per feature:"}]},{"t":"ol","start":1,"items":[[{"text":"Requests per day."}],[{"text":"Input tokens per request, from the budget ledger in module one."}],[{"text":"Output tokens per request, at the 95th percentile rather than the mean."}],[{"text":"Expected cache hit rate: assume 0 until measured."}]]},{"t":"p","runs":[{"text":"Multiply, then double it. The doubling is not pessimism; it covers retries, evaluation runs, the internal team’s usage, and the requests that turn out to be four calls rather than one because a tool was invoked."}]},{"t":"p","runs":[{"text":"A forecast that comes in at twice the eventual bill costs you a conversation. One that comes in at half costs you the project’s credibility in month two."}]}],"check":{"question":[{"text":"A ten-turn conversation with 500 tokens added per turn is billed for roughly 27,500 input tokens rather than 5,000. Why?"}],"options":[{"text":[{"text":"Every turn resends the full prior history, so the billed total is the sum of a growing prefix."}],"correct":true,"tempting":null},{"text":[{"text":"Output tokens are billed at a higher rate, and ten turns of output accumulate."}],"correct":false,"tempting":[{"text":"Output is indeed dearer, but the figure quoted is input tokens, and the ratio described is about resending rather than rate."}]},{"text":[{"text":"The chat template adds delimiters to every message, multiplying the count."}],"correct":false,"tempting":[{"text":"Template overhead is real but is a few tokens per message, nowhere near a fivefold increase."}]},{"text":[{"text":"Prefix caching writes each prefix at a premium, inflating the counted input."}],"correct":false,"tempting":[{"text":"Cache writes carry a price premium, not a token multiplier, and caching reduces the effective cost of resent history."}]}],"order":[2,1,3,0],"why":null},"prev":{"slug":"attention-sinks-and-the-first-tokens","title":"Why the first tokens are special","minutes":8},"next":{"slug":"prefill-decode-and-where-the-time-goes","title":"Where the milliseconds go","minutes":9}}}