{"course":{"slug":"context-engineering","title":"Context Engineering","url":"https://addaly.com/learn/context-engineering","level":"advanced","track":"build"},"lesson":{"slug":"a-conversation-is-a-growing-prefix","title":"Every turn carries all the previous ones","url":"https://addaly.com/learn/context-engineering/a-conversation-is-a-growing-prefix","position":49,"of":87,"minutes":8,"answer":"A conversation is your code resending the whole transcript every turn, which makes the bill quadratic, dilutes the original instruction, and turns an early model error into a confident fact the later turns build on.","keyIdea":"A conversation is your code resending the whole transcript every turn, which makes the bill quadratic, dilutes the original instruction, and turns an early model error into a confident fact the later turns build on."},"body":{"format":"markdown","text":"## The shape of the problem\n\nA model has no memory between calls. What looks like a conversation is your code resending the whole transcript on every turn, plus one new message. That single mechanical fact produces four consequences, and every technique in this module is a response to one of them.\n\n**The bill grows with the square.** Turn ten resends turns one through nine. The cost lesson did the arithmetic: ten turns of 500 tokens is 27,500 billed input tokens, not 5,000.\n\n> Figure: Ten turns of five hundred tokens each. Line chart of cumulative input tokens billed against turn number: Billed: the transcript resent every turn (1, 500) (2, 1500) (3, 3000) (4, 5000) (5, 7500) (6, 10500) (7, 14000) (8, 18000) (9, 22500) (10, 27500); What linear growth would have cost (1, 500) (2, 1000) (3, 1500) (4, 2000) (5, 2500) (6, 3000) (7, 3500) (8, 4000) (9, 4500) (10, 5000). The model has no memory between calls, so every turn resends the whole transcript plus one new message. Five thousand tokens of conversation bills twenty-seven and a half thousand input tokens. A team that plans for the lower line underestimates a twenty-turn conversation by roughly ten times, and every quality complaint lives in the same tail as the cost.\n\n**The window fills.** At some point the transcript exceeds the history budget, and something has to give. If you have not decided what, a framework will decide for you, usually by dropping from the front.\n\n**Attention dilutes.** By turn twenty the user's original instruction is thousands of tokens from where generation begins, competing with everything since. The position module explains why this weakens it.\n\n**Errors persist.** This is the one people underestimate. A wrong statement the model made on turn three is, on turn nine, part of the context: sitting in the assistant role, phrased confidently, indistinguishable from an established fact. Models are consistent with their own prior output, so the error does not just survive, it gets built on. A conversation that has gone wrong tends to stay wrong, and adding a correction later is weaker than the original mistake because the mistake has been repeated and elaborated.\n\n## Where the growth actually comes from\n\nMeasure it on your own traffic before optimising. The three sources are unequal:\n\n- **Assistant turns** are usually the largest. They are longer than user turns and there are just as many of them. A verbose assistant is a compounding cost.\n- **Tool results** are spiky. Most turns add nothing; one search result adds 12,000 tokens and permanently changes the shape of the conversation.\n- **User turns** are the smallest and the most variable: three words or a pasted 40-page document.\n\nSo the highest-return length control is often on the assistant's own output, which is also the expensive token rate. \"Be concise unless asked otherwise\" is a cost lever, a latency lever and a context lever at once.\n\n## The numbers you need from your logs\n\nFour, and they take an hour to extract:\n\n1. **Median and 95th percentile turns per conversation.** Almost always a long tail: most conversations are two turns and some are ninety.\n2. **Median and 95th percentile tokens per turn**, split by role.\n3. **The distribution of total conversation tokens.** This decides whether compaction is a real requirement or a hypothetical one.\n4. **What fraction of conversations exceed your history budget at all.** If it is 2%, you need a correct-but-simple policy. If it is 40%, compaction is a core feature and deserves design.\n\nTeams commonly discover that the 95th-percentile conversation is ten times the median, and that all of their cost and all of their quality complaints live in that tail.\n\n## Set the history budget as a cap, not a hope\n\nFrom the budget ledger in module one: history gets a number. When the transcript exceeds it, a policy runs. The policy is the subject of the next three lessons, and the important part is that it exists and is yours.\n\nOne rule to fix now, because it is where naive implementations break: **the system prompt is not part of the history window.** Pin it outside. A `messages[-20:]` that includes the system message removes it on turn twenty-one, and the assistant changes personality mid-conversation with no error anywhere.\n\n## Two things worth doing before compaction\n\nBefore building summarisation machinery, two cheaper interventions often solve the problem.\n\n**Cache the history prefix.** History is append-only, so everything up to the last turn is a stable prefix. A cache breakpoint after it turns the quadratic bill into something much flatter without changing behaviour at all. This is the single highest-value change for a chat product and it requires no summarisation.\n\n**Drop tool results that have been used.** A search result that produced an answer three turns ago is usually dead weight. Replacing it with a one-line note (\"searched for X, 8 results, used [3] and [5]\") recovers most of a large block with almost no loss. This is compaction, but of the easiest kind, on the material with the worst value-per-token in the whole transcript.\n\nDo both first. Then measure again, and see whether you still need the harder thing."},"check":{"question":"A model states something incorrect on turn three. By turn twelve it is repeating and elaborating the error even after a user correction. What is the mechanism?","options":[{"text":"The user's correction was placed in the user role, which the model weights less than its own assistant turns.","correct":false,"tempting":"Role weighting favours system and recent content; it does not rank assistant output above user instruction."},{"text":"The error is now context in the assistant role and has been repeated, so the model is being consistent with what the window says it believes.","correct":true},{"text":"History compaction summarised the error into a fact, giving it more authority than the correction.","correct":false,"tempting":"Would explain it in a system that compacts, but the effect appears in plain uncompacted transcripts too."},{"text":"The correction was too far from the point of generation to influence the output.","correct":false,"tempting":"A recent correction sits close to generation; distance would predict the opposite of what happens."}]},"prev":{"slug":"denial-of-wallet-and-context-bombs","title":"How one user spends your whole budget","url":"https://addaly.com/learn/context-engineering/denial-of-wallet-and-context-bombs"},"next":{"slug":"what-to-keep-verbatim","title":"Which turns are worth their tokens","url":"https://addaly.com/learn/context-engineering/what-to-keep-verbatim"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Every turn carries all the previous ones', in Context Engineering. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/context-engineering/a-conversation-is-a-growing-prefix (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Every turn carries all the previous ones","courseTitle":"Context Engineering","module":{"index":6,"slug":"conversation-and-memory","title":"Conversation, memory and compaction"},"answer":null,"keyIdea":"A conversation is your code resending the whole transcript every turn, which makes the bill quadratic, dilutes the original instruction, and turns an early model error into a confident fact the later turns build on.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure 6.1","figure":{"type":"line","title":"Ten turns of five hundred tokens each","series":[{"label":"Billed: the transcript resent every turn","points":[[1,500],[2,1500],[3,3000],[4,5000],[5,7500],[6,10500],[7,14000],[8,18000],[9,22500],[10,27500]]},{"label":"What linear growth would have cost","points":[[1,500],[2,1000],[3,1500],[4,2000],[5,2500],[6,3000],[7,3500],[8,4000],[9,4500],[10,5000]]}],"xLabel":"turn number","yLabel":"cumulative input tokens billed","caption":"The model has no memory between calls, so every turn resends the whole transcript plus one new message. Five thousand tokens of conversation bills twenty-seven and a half thousand input tokens. A team that plans for the lower line underestimates a twenty-turn conversation by roughly ten times, and every quality complaint lives in the same tail as the cost."},"alt":"Figure: Ten turns of five hundred tokens each. Line chart of cumulative input tokens billed against turn number: Billed: the transcript resent every turn (1, 500) (2, 1500) (3, 3000) (4, 5000) (5, 7500) (6, 10500) (7, 14000) (8, 18000) (9, 22500) (10, 27500); What linear growth would have cost (1, 500) (2, 1000) (3, 1500) (4, 2000) (5, 2500) (6, 3000) (7, 3500) (8, 4000) (9, 4500) (10, 5000). The model has no memory between calls, so every turn resends the whole transcript plus one new message. Five thousand tokens of conversation bills twenty-seven and a half thousand input tokens. A team that plans for the lower line underestimates a twenty-turn conversation by roughly ten times, and every quality complaint lives in the same tail as the cost."}},"blocks":[{"t":"h2","id":"the-shape-of-the-problem","runs":[{"text":"The shape of the problem"}]},{"t":"p","runs":[{"text":"A model has no memory between calls. What looks like a conversation is your code resending the whole transcript on every turn, plus one new message. That single mechanical fact produces four consequences, and every technique in this module is a response to one of them."}]},{"t":"p","runs":[{"text":"The bill grows with the square.","b":true},{"text":" Turn ten resends turns one through nine. The cost lesson did the arithmetic: ten turns of 500 tokens is 27,500 billed input tokens, not 5,000."}]},{"t":"p","runs":[{"text":"The window fills.","b":true},{"text":" At some point the transcript exceeds the history budget, and something has to give. If you have not decided what, a framework will decide for you, usually by dropping from the front."}]},{"t":"p","runs":[{"text":"Attention dilutes.","b":true},{"text":" By turn twenty the user’s original instruction is thousands of tokens from where generation begins, competing with everything since. The position module explains why this weakens it."}]},{"t":"p","runs":[{"text":"Errors persist.","b":true},{"text":" This is the one people underestimate. A wrong statement the model made on turn three is, on turn nine, part of the context: sitting in the assistant role, phrased confidently, indistinguishable from an established fact. Models are consistent with their own prior output, so the error does not just survive, it gets built on. A conversation that has gone wrong tends to stay wrong, and adding a correction later is weaker than the original mistake because the mistake has been repeated and elaborated."}]},{"t":"h2","id":"where-the-growth-actually-comes-from","runs":[{"text":"Where the growth actually comes from"}]},{"t":"p","runs":[{"text":"Measure it on your own traffic before optimising. The three sources are unequal:"}]},{"t":"ul","items":[[{"text":"Assistant turns","b":true},{"text":" are usually the largest. They are longer than user turns and there are just as many of them. A verbose assistant is a compounding cost."}],[{"text":"Tool results","b":true},{"text":" are spiky. Most turns add nothing; one search result adds 12,000 tokens and permanently changes the shape of the conversation."}],[{"text":"User turns","b":true},{"text":" are the smallest and the most variable: three words or a pasted 40-page document."}]]},{"t":"p","runs":[{"text":"So the highest-return length control is often on the assistant’s own output, which is also the expensive token rate. “Be concise unless asked otherwise” is a cost lever, a latency lever and a context lever at once."}]},{"t":"h2","id":"the-numbers-you-need-from-your-logs","runs":[{"text":"The numbers you need from your logs"}]},{"t":"p","runs":[{"text":"Four, and they take an hour to extract:"}]},{"t":"ol","start":1,"items":[[{"text":"Median and 95th percentile turns per conversation.","b":true},{"text":" Almost always a long tail: most conversations are two turns and some are ninety."}],[{"text":"Median and 95th percentile tokens per turn","b":true},{"text":", split by role."}],[{"text":"The distribution of total conversation tokens.","b":true},{"text":" This decides whether compaction is a real requirement or a hypothetical one."}],[{"text":"What fraction of conversations exceed your history budget at all.","b":true},{"text":" If it is 2%, you need a correct-but-simple policy. If it is 40%, compaction is a core feature and deserves design."}]]},{"t":"p","runs":[{"text":"Teams commonly discover that the 95th-percentile conversation is ten times the median, and that all of their cost and all of their quality complaints live in that tail."}]},{"t":"h2","id":"set-the-history-budget-as-a-cap-not-a-hope","runs":[{"text":"Set the history budget as a cap, not a hope"}]},{"t":"p","runs":[{"text":"From the budget ledger in module one: history gets a number. When the transcript exceeds it, a policy runs. The policy is the subject of the next three lessons, and the important part is that it exists and is yours."}]},{"t":"p","runs":[{"text":"One rule to fix now, because it is where naive implementations break: "},{"text":"the system prompt is not part of the history window.","b":true},{"text":" Pin it outside. A "},{"text":"messages[-20:]","code":true},{"text":" that includes the system message removes it on turn twenty-one, and the assistant changes personality mid-conversation with no error anywhere."}]},{"t":"h2","id":"two-things-worth-doing-before-compaction","runs":[{"text":"Two things worth doing before compaction"}]},{"t":"p","runs":[{"text":"Before building summarisation machinery, two cheaper interventions often solve the problem."}]},{"t":"p","runs":[{"text":"Cache the history prefix.","b":true},{"text":" History is append-only, so everything up to the last turn is a stable prefix. A cache breakpoint after it turns the quadratic bill into something much flatter without changing behaviour at all. This is the single highest-value change for a chat product and it requires no summarisation."}]},{"t":"p","runs":[{"text":"Drop tool results that have been used.","b":true},{"text":" A search result that produced an answer three turns ago is usually dead weight. Replacing it with a one-line note (“searched for X, 8 results, used [3] and [5]”) recovers most of a large block with almost no loss. This is compaction, but of the easiest kind, on the material with the worst value-per-token in the whole transcript."}]},{"t":"p","runs":[{"text":"Do both first. Then measure again, and see whether you still need the harder thing."}]}],"check":{"question":[{"text":"A model states something incorrect on turn three. By turn twelve it is repeating and elaborating the error even after a user correction. What is the mechanism?"}],"options":[{"text":[{"text":"The user’s correction was placed in the user role, which the model weights less than its own assistant turns."}],"correct":false,"tempting":[{"text":"Role weighting favours system and recent content; it does not rank assistant output above user instruction."}]},{"text":[{"text":"The error is now context in the assistant role and has been repeated, so the model is being consistent with what the window says it believes."}],"correct":true,"tempting":null},{"text":[{"text":"History compaction summarised the error into a fact, giving it more authority than the correction."}],"correct":false,"tempting":[{"text":"Would explain it in a system that compacts, but the effect appears in plain uncompacted transcripts too."}]},{"text":[{"text":"The correction was too far from the point of generation to influence the output."}],"correct":false,"tempting":[{"text":"A recent correction sits close to generation; distance would predict the opposite of what happens."}]}],"order":[2,3,0,1],"why":null},"prev":{"slug":"denial-of-wallet-and-context-bombs","title":"How one user spends your whole budget","minutes":9},"next":{"slug":"what-to-keep-verbatim","title":"Which turns are worth their tokens","minutes":8}}}