{"course":{"slug":"building-with-ai","title":"Building with AI","url":"https://addaly.com/learn/building-with-ai","level":"intermediate","track":"build"},"lesson":{"slug":"the-cost-model-of-a-feature","title":"The cost model of a feature, line by line","url":"https://addaly.com/learn/building-with-ai/the-cost-model-of-a-feature","position":71,"of":80,"minutes":9,"answer":"A feature's cost is calls per user per day times tokens per call, split into cached input, uncached input and output at their three different prices, plus the fixed costs of indexing and evaluation.","keyIdea":"A feature's cost is calls per user per day times tokens per call, split into cached input, uncached input and output at their three different prices, plus the fixed costs of indexing and evaluation, and once written as a spreadsheet, one variable usually dominates and that is the one to engineer."},"body":{"format":"markdown","text":"## From a call to a feature\n\nThe first module priced one request: input tokens at one rate, output at a higher one, cached input at a discount. A feature is that arithmetic multiplied by behaviour (how often people use it, how long their sessions run, how much context each turn carries) plus costs that do not scale with users at all. Write it as a spreadsheet before the feature exists, with guessed numbers, and replace each guess with a measurement from the traces as they arrive. The free path is any spreadsheet; the discipline is filling in every cell.\n\n## The variables\n\nPer turn: uncached input tokens, cached input tokens, output tokens. Per session: turns. Per user: sessions per day. Then the population: active users per day. And three prices, from the provider's page: uncached input, cached input, output.\n\nTurn cost is the sum of the three token counts times their prices. Multiply up.\n\n## A worked example\n\nA support assistant over product documentation. Ten thousand active users; each has three sessions a week, so about 4,300 sessions a day; six turns a session. Each turn sends a 1,500-token system prompt and retrieved context that is stable across the session, so cached after the first turn; 1,000 tokens of conversation and new retrieval that are not cached; and receives 300 tokens of answer. Prices: $3 per million uncached input, $0.30 cached, $15 output.\n\nPer turn, after the first in a session: cached 1,500 tokens at $0.30 per million is $0.00045; uncached 1,000 at $3 is $0.003; output 300 at $15 is $0.0045. About $0.008. The first turn of a session pays full price on the prefix, adding about $0.004 once per session.\n\nPer session: six turns at $0.008 plus the first-turn premium, roughly $0.05. Per day: 4,300 sessions is about $220. Per month: around $6,500. Per user per month: sixty-five cents.\n\n## What the example teaches\n\nLook at where the money went. Output tokens were more than half of each turn's cost, at 300 tokens. Uncached input was most of the rest. The cached prefix, which is the largest block of tokens in the request, was almost free. Three consequences follow, and they generalise.\n\nOutput length is the first lever. A prompt change that trims answers from 300 tokens to 200 saves a third of the output cost, which is nearly a fifth of the whole feature. That change costs nothing and it is the reason the harness in the evaluation module tracks output length as a regression.\n\nCache hit rate is the second. The example assumed the prefix was cached on every turn after the first. If sessions are far apart and the cache expires between them, or the prefix changes because retrieved context is placed before the system prompt, the cached line becomes uncached and the turn cost rises by half. The prompt-caching lesson's ordering rule is worth real money here.\n\nModel choice is the third, and it is multiplicative on everything. The same feature on a model at a tenth of the price costs a tenth, if the evaluation set says it is good enough. The routing lesson later in this block is about paying the high price only where it is needed.\n\n## The costs that do not scale with users\n\nIngestion and embedding of the corpus: small, from the retrieval module, and repeated on re-index. The evaluation harness: tens of dollars a month. The judge on shadow-mode comparisons: more, during a change. Human review of moderation and feedback queues: a person's hours, and the largest fixed cost most teams have. Write these down too; a feature at a thousand users is dominated by them, and at a hundred thousand they vanish.\n\n> Figure: Where a turn's money actually goes. Bars (dollars per 1,000 turns): Output: 300 tokens at $15/M 4.5; Uncached input: 1,000 tokens at $3/M 3; Cached prefix: 1,500 tokens at $0.30/M 0.45. The largest block of tokens in the request is nearly free and the smallest is more than half the bill. That is why trimming answers from 300 tokens to 200 saves almost a fifth of the whole feature, and why the harness treats a rise in output length as a regression.\n\n## Sensitivity\n\nChange one variable at a time by half and double, and watch the monthly figure. Whichever moves it most is the one to instrument first and engineer first. In the example it is output length, then cache hit rate, then turns per session. In a document-analysis feature it is input length. In an inline completion feature, from the interface module, it is calls per user, by a wide margin.\n\n## Margin, and the price of the feature\n\nIf the feature is charged for, the per-user monthly cost is the floor of the price. If it is free, it is a marketing expense with a known size, and the question is what it earns in retention or conversion. Either way the number should be on a page someone reviews monthly against the traces, because the guesses that built the spreadsheet will be wrong in both directions, and the day the actual figure is twice the model is the day to find out which cell was wrong."},"check":{"question":"In a support assistant, each turn sends 1,500 cached tokens, 1,000 uncached tokens and receives 300 output tokens, at $0.30, $3 and $15 per million respectively. Which component dominates the cost of a turn?","options":[{"text":"The cached prefix, because at 1,500 tokens it is the largest block in the request and is sent on every turn.","correct":false,"tempting":"Size misleads here; the cached rate is a tenth of the uncached one, so the largest block is nearly the cheapest line."},{"text":"The uncached input, because it is charged at ten times the cached rate.","correct":false,"tempting":"It is the second-largest line, at $0.003, but output at $0.0045 exceeds it."},{"text":"The output, because 300 tokens at $15 per million exceeds both input lines.","correct":true},{"text":"None dominates; the three lines are within a few per cent of each other.","correct":false,"tempting":"The arithmetic gives roughly $0.00045, $0.003 and $0.0045, which is a tenfold spread."}]},"prev":{"slug":"disclosure-consent-and-the-law-as-it-stands","title":"Disclosure, consent and the law as it stands","url":"https://addaly.com/learn/building-with-ai/disclosure-consent-and-the-law-as-it-stands"},"next":{"slug":"where-the-milliseconds-go","title":"Where the milliseconds go","url":"https://addaly.com/learn/building-with-ai/where-the-milliseconds-go"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The cost model of a feature, line by line', in Building with AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/building-with-ai/the-cost-model-of-a-feature (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The cost model of a feature, line by line","courseTitle":"Building with AI","module":{"index":8,"slug":"cost-latency-and-running-it","title":"Cost, latency and running it in production"},"answer":null,"keyIdea":"A feature’s cost is calls per user per day times tokens per call, split into cached input, uncached input and output at their three different prices, plus the fixed costs of indexing and evaluation, and once written as a spreadsheet, one variable usually dominates and that is the one to engineer.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure 8.1","figure":{"type":"bars","unit":"dollars per 1,000 turns","title":"Where a turn’s money actually goes","series":[{"label":"Output: 300 tokens at $15/M","value":4.5,"highlight":true},{"label":"Uncached input: 1,000 tokens at $3/M","value":3},{"label":"Cached prefix: 1,500 tokens at $0.30/M","value":0.45}],"caption":"The largest block of tokens in the request is nearly free and the smallest is more than half the bill. That is why trimming answers from 300 tokens to 200 saves almost a fifth of the whole feature, and why the harness treats a rise in output length as a regression."},"alt":"Figure: Where a turn’s money actually goes. Bars (dollars per 1,000 turns): Output: 300 tokens at $15/M 4.5; Uncached input: 1,000 tokens at $3/M 3; Cached prefix: 1,500 tokens at $0.30/M 0.45. The largest block of tokens in the request is nearly free and the smallest is more than half the bill. That is why trimming answers from 300 tokens to 200 saves almost a fifth of the whole feature, and why the harness treats a rise in output length as a regression."}},"blocks":[{"t":"h2","id":"from-a-call-to-a-feature","runs":[{"text":"From a call to a feature"}]},{"t":"p","runs":[{"text":"The first module priced one request: input tokens at one rate, output at a higher one, cached input at a discount. A feature is that arithmetic multiplied by behaviour (how often people use it, how long their sessions run, how much context each turn carries) plus costs that do not scale with users at all. Write it as a spreadsheet before the feature exists, with guessed numbers, and replace each guess with a measurement from the traces as they arrive. The free path is any spreadsheet; the discipline is filling in every cell."}]},{"t":"h2","id":"the-variables","runs":[{"text":"The variables"}]},{"t":"p","runs":[{"text":"Per turn: uncached input tokens, cached input tokens, output tokens. Per session: turns. Per user: sessions per day. Then the population: active users per day. And three prices, from the provider’s page: uncached input, cached input, output."}]},{"t":"p","runs":[{"text":"Turn cost is the sum of the three token counts times their prices. Multiply up."}]},{"t":"h2","id":"a-worked-example","runs":[{"text":"A worked example"}]},{"t":"p","runs":[{"text":"A support assistant over product documentation. Ten thousand active users; each has three sessions a week, so about 4,300 sessions a day; six turns a session. Each turn sends a 1,500-token system prompt and retrieved context that is stable across the session, so cached after the first turn; 1,000 tokens of conversation and new retrieval that are not cached; and receives 300 tokens of answer. Prices: $3 per million uncached input, $0.30 cached, $15 output."}]},{"t":"p","runs":[{"text":"Per turn, after the first in a session: cached 1,500 tokens at $0.30 per million is $0.00045; uncached 1,000 at $3 is $0.003; output 300 at $15 is $0.0045. About $0.008. The first turn of a session pays full price on the prefix, adding about $0.004 once per session."}]},{"t":"p","runs":[{"text":"Per session: six turns at $0.008 plus the first-turn premium, roughly $0.05. Per day: 4,300 sessions is about $220. Per month: around $6,500. Per user per month: sixty-five cents."}]},{"t":"h2","id":"what-the-example-teaches","runs":[{"text":"What the example teaches"}]},{"t":"p","runs":[{"text":"Look at where the money went. Output tokens were more than half of each turn’s cost, at 300 tokens. Uncached input was most of the rest. The cached prefix, which is the largest block of tokens in the request, was almost free. Three consequences follow, and they generalise."}]},{"t":"p","runs":[{"text":"Output length is the first lever. A prompt change that trims answers from 300 tokens to 200 saves a third of the output cost, which is nearly a fifth of the whole feature. That change costs nothing and it is the reason the harness in the evaluation module tracks output length as a regression."}]},{"t":"p","runs":[{"text":"Cache hit rate is the second. The example assumed the prefix was cached on every turn after the first. If sessions are far apart and the cache expires between them, or the prefix changes because retrieved context is placed before the system prompt, the cached line becomes uncached and the turn cost rises by half. The prompt-caching lesson’s ordering rule is worth real money here."}]},{"t":"p","runs":[{"text":"Model choice is the third, and it is multiplicative on everything. The same feature on a model at a tenth of the price costs a tenth, if the evaluation set says it is good enough. The routing lesson later in this block is about paying the high price only where it is needed."}]},{"t":"h2","id":"the-costs-that-do-not-scale-with-users","runs":[{"text":"The costs that do not scale with users"}]},{"t":"p","runs":[{"text":"Ingestion and embedding of the corpus: small, from the retrieval module, and repeated on re-index. The evaluation harness: tens of dollars a month. The judge on shadow-mode comparisons: more, during a change. Human review of moderation and feedback queues: a person’s hours, and the largest fixed cost most teams have. Write these down too; a feature at a thousand users is dominated by them, and at a hundred thousand they vanish."}]},{"t":"h2","id":"sensitivity","runs":[{"text":"Sensitivity"}]},{"t":"p","runs":[{"text":"Change one variable at a time by half and double, and watch the monthly figure. Whichever moves it most is the one to instrument first and engineer first. In the example it is output length, then cache hit rate, then turns per session. In a document-analysis feature it is input length. In an inline completion feature, from the interface module, it is calls per user, by a wide margin."}]},{"t":"h2","id":"margin-and-the-price-of-the-feature","runs":[{"text":"Margin, and the price of the feature"}]},{"t":"p","runs":[{"text":"If the feature is charged for, the per-user monthly cost is the floor of the price. If it is free, it is a marketing expense with a known size, and the question is what it earns in retention or conversion. Either way the number should be on a page someone reviews monthly against the traces, because the guesses that built the spreadsheet will be wrong in both directions, and the day the actual figure is twice the model is the day to find out which cell was wrong."}]}],"check":{"question":[{"text":"In a support assistant, each turn sends 1,500 cached tokens, 1,000 uncached tokens and receives 300 output tokens, at $0.30, $3 and $15 per million respectively. Which component dominates the cost of a turn?"}],"options":[{"text":[{"text":"The cached prefix, because at 1,500 tokens it is the largest block in the request and is sent on every turn."}],"correct":false,"tempting":[{"text":"Size misleads here; the cached rate is a tenth of the uncached one, so the largest block is nearly the cheapest line."}]},{"text":[{"text":"The uncached input, because it is charged at ten times the cached rate."}],"correct":false,"tempting":[{"text":"It is the second-largest line, at $0.003, but output at $0.0045 exceeds it."}]},{"text":[{"text":"The output, because 300 tokens at $15 per million exceeds both input lines."}],"correct":true,"tempting":null},{"text":[{"text":"None dominates; the three lines are within a few per cent of each other."}],"correct":false,"tempting":[{"text":"The arithmetic gives roughly $0.00045, $0.003 and $0.0045, which is a tenfold spread."}]}],"order":[3,0,1,2],"why":null},"prev":{"slug":"disclosure-consent-and-the-law-as-it-stands","title":"Disclosure, consent and the law as it stands","minutes":10},"next":{"slug":"where-the-milliseconds-go","title":"Where the milliseconds go","minutes":9}}}