{"course":{"slug":"context-engineering","title":"Context Engineering","url":"https://addaly.com/learn/context-engineering","level":"advanced","track":"build"},"lesson":{"slug":"what-to-keep-verbatim","title":"Which turns are worth their tokens","url":"https://addaly.com/learn/context-engineering/what-to-keep-verbatim","position":50,"of":87,"minutes":8,"answer":"Value in a transcript is not correlated with recency, so pin the task statement and the constraints verbatim at the head, keep the last handful of turns verbatim at the tail, compress the middle.","keyIdea":"Value in a transcript is not correlated with recency, so pin the task statement and the constraints verbatim at the head, keep the last handful of turns verbatim at the tail, compress the middle, and stub out spent tool results, while keeping the full transcript in storage regardless of what the window holds."},"body":{"format":"markdown","text":"## Turns are not equal, and a window treats them as if they are\n\nA sliding window keeps the last N messages. It is simple, it is what every framework does by default, and it throws away the wrong things, because the value of a turn has almost nothing to do with how recent it is.\n\nA better policy comes from sorting the transcript into four kinds.\n\n**Keep verbatim, always.** The exact words matter and no paraphrase substitutes:\n\n- The **task statement**, usually the first substantive user message. It is the thing everything else serves and it is what a tail window loses first.\n- Anything the user explicitly asked to be remembered or used exactly: a name, a specification, a piece of text to work from.\n- **Constraints and decisions**: \"use British spelling\", \"the budget is 40,000\", \"we ruled out option B\".\n- **Exact strings**: identifiers, code, file paths, quoted material. A summariser will round `ORD-88213` to \"the order\" and the specificity is gone for good.\n- **The last few turns**, because the immediate thread of the conversation is what the next reply continues.\n\n**Compress.** The content mattered and the wording did not: long explanations the user accepted, tool results already acted on, an earlier draft that has been superseded, a document read three turns ago.\n\n**Drop entirely.** Greetings, acknowledgements, \"thanks, that's helpful\", the model's own restatements of the question, and the apology it produced when it got something wrong. This is more of a transcript than people expect: often 10 to 20% of it.\n\n**Never keep.** Anything the user asked you to forget, and anything that should not have been retained under your own retention policy. The forgetting lesson later in this module treats this properly.\n\n## The shape that follows\n\nMost systems that handle long conversations well converge on the same layout:\n\n```\n[system prompt]                       pinned, outside the window\n[structured state]                    facts as fields, next lesson\n[first user message]                  pinned verbatim\n[compacted middle]                    a summary of turns 2..n-k\n[last k turns]                        verbatim, k typically 4-8\n[current message]\n```\n\nHead and tail verbatim, middle compressed. This is not a coincidence: it is the same U as the position module, arrived at from a different direction. The beginning and the end are where material is used, so those are the parts worth their full token cost.\n\n> Figure: The layout long conversations converge on. Layers, top to bottom: System prompt (Pinned outside the history window. A slice of the last twenty messages that includes it deletes it on turn twenty-one); Structured state (Ids, amounts, constraints, decisions and rejections as named fields rather than sentences); The task statement (Verbatim. The first substantive user message, which everything else serves); The compressed middle (Explanations already accepted, superseded drafts, a document read three turns ago); Spent tool results, stubbed (Thirty tokens instead of twelve thousand, with an id so it can be fetched again); The last four to eight turns (Verbatim, because the immediate thread is what the next reply has to be coherent with); The new message (And the full transcript is in storage regardless of what the window holds). Head and tail verbatim, middle compressed, the same U as the position module, arrived at from a different direction. Whatever the policy drops from the window stays in storage: the window is a working set, the transcript is the record, and conflating them turns every context decision into a permanent data-loss decision.\n\n## Choosing k\n\nThe number of recent turns to keep verbatim is a real parameter and it is worth measuring rather than guessing.\n\nToo small and the model loses the immediate thread: it asks a question that was answered two turns ago, or re-explains something the user already rejected. Too large and the middle gets compacted aggressively to make room, and the conversation loses its longer arc.\n\nFour to eight turns is the usual landing zone. Test it the same way as everything else: take real conversations that went wrong, replay them at k = 2, 4, 8, and see where the failure disappears.\n\n## Compress tool results first\n\nIf you do only one thing from this lesson, do this. Tool results have the worst value-per-token in the transcript by a wide margin: large, structured, and usually consumed entirely within one turn.\n\nReplace a spent result with a stub:\n\n```\n[tool: search_orders — 8 results for \"late delivery\", used ORD-88213 and\n ORD-88220; full result retained as result_id=r_4471]\n```\n\nThirty tokens instead of twelve thousand, and the `result_id` means the agent can fetch it again if it turns out to be needed. The next module's lesson on offloading to files generalises this pattern.\n\n## The audit that finds the waste\n\nTake five long conversations and classify every turn into the four kinds above, with a token count beside each. The proportions are usually startling: acknowledgements and restatements at 15%, spent tool results at 40%, and the constraints that actually govern the task at under 2%.\n\nThat single count tells you where the policy should bite, and it almost always says the same thing: the cheapest large win is stubbing tool results, and the most valuable small win is pinning the two per cent that governs everything.\n\n## Do not delete the transcript you compacted\n\nWhatever your policy discards from the window, keep in storage. Three reasons: the user may refer to it (\"what did you say about the deadline?\"), you may need to re-derive a better summary later, and a support investigation into a bad answer needs the raw material rather than your compression of it.\n\nThe window is a working set. The transcript is the record. Conflating them means every context decision is also a permanent data-loss decision, which is a much heavier thing to get right."},"check":{"question":"A support assistant using a 20-message sliding window starts giving answers unrelated to the customer's original problem in long conversations. What is the most likely cause?","options":[{"text":"Attention dilution means the original message is present but too distant to be used.","correct":false,"tempting":"A real effect at long context, but it degrades use gradually rather than producing answers unrelated to the problem."},{"text":"The first message, containing the task, has slid out of the window entirely.","correct":true},{"text":"Tool results have accumulated and are crowding out the recent turns.","correct":false,"tempting":"Would push out recent context rather than the oldest, and the window keeps recent messages by construction."},{"text":"The model is being consistent with its own earlier errors carried forward in the transcript.","correct":false,"tempting":"Error carry-forward produces persistently wrong answers about the right problem, not answers about a different one."}]},"prev":{"slug":"a-conversation-is-a-growing-prefix","title":"Every turn carries all the previous ones","url":"https://addaly.com/learn/context-engineering/a-conversation-is-a-growing-prefix"},"next":{"slug":"compaction-when-and-how","title":"Compacting without losing the thread","url":"https://addaly.com/learn/context-engineering/compaction-when-and-how"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Which turns are worth their tokens', in Context Engineering. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/context-engineering/what-to-keep-verbatim (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Which turns are worth their tokens","courseTitle":"Context Engineering","module":{"index":6,"slug":"conversation-and-memory","title":"Conversation, memory and compaction"},"answer":null,"keyIdea":"Value in a transcript is not correlated with recency, so pin the task statement and the constraints verbatim at the head, keep the last handful of turns verbatim at the tail, compress the middle, and stub out spent tool results, while keeping the full transcript in storage regardless of what the window holds.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"stack","title":"The layout long conversations converge on","layers":[{"label":"System prompt","detail":"Pinned outside the history window. A slice of the last twenty messages that includes it deletes it on turn twenty-one"},{"label":"Structured state","detail":"Ids, amounts, constraints, decisions and rejections as named fields rather than sentences"},{"label":"The task statement","detail":"Verbatim. The first substantive user message, which everything else serves"},{"label":"The compressed middle","detail":"Explanations already accepted, superseded drafts, a document read three turns ago"},{"label":"Spent tool results, stubbed","detail":"Thirty tokens instead of twelve thousand, with an id so it can be fetched again"},{"label":"The last four to eight turns","detail":"Verbatim, because the immediate thread is what the next reply has to be coherent with"},{"label":"The new message","detail":"And the full transcript is in storage regardless of what the window holds"}],"caption":"Head and tail verbatim, middle compressed, the same U as the position module, arrived at from a different direction. Whatever the policy drops from the window stays in storage: the window is a working set, the transcript is the record, and conflating them turns every context decision into a permanent data-loss decision."},"alt":"Figure: The layout long conversations converge on. Layers, top to bottom: System prompt (Pinned outside the history window. A slice of the last twenty messages that includes it deletes it on turn twenty-one); Structured state (Ids, amounts, constraints, decisions and rejections as named fields rather than sentences); The task statement (Verbatim. The first substantive user message, which everything else serves); The compressed middle (Explanations already accepted, superseded drafts, a document read three turns ago); Spent tool results, stubbed (Thirty tokens instead of twelve thousand, with an id so it can be fetched again); The last four to eight turns (Verbatim, because the immediate thread is what the next reply has to be coherent with); The new message (And the full transcript is in storage regardless of what the window holds). Head and tail verbatim, middle compressed, the same U as the position module, arrived at from a different direction. Whatever the policy drops from the window stays in storage: the window is a working set, the transcript is the record, and conflating them turns every context decision into a permanent data-loss decision."}},"blocks":[{"t":"h2","id":"turns-are-not-equal-and-a-window-treats-them-as-if-they-are","runs":[{"text":"Turns are not equal, and a window treats them as if they are"}]},{"t":"p","runs":[{"text":"A sliding window keeps the last N messages. It is simple, it is what every framework does by default, and it throws away the wrong things, because the value of a turn has almost nothing to do with how recent it is."}]},{"t":"p","runs":[{"text":"A better policy comes from sorting the transcript into four kinds."}]},{"t":"p","runs":[{"text":"Keep verbatim, always.","b":true},{"text":" The exact words matter and no paraphrase substitutes:"}]},{"t":"ul","items":[[{"text":"The "},{"text":"task statement","b":true},{"text":", usually the first substantive user message. It is the thing everything else serves and it is what a tail window loses first."}],[{"text":"Anything the user explicitly asked to be remembered or used exactly: a name, a specification, a piece of text to work from."}],[{"text":"Constraints and decisions","b":true},{"text":": “use British spelling”, “the budget is 40,000”, “we ruled out option B”."}],[{"text":"Exact strings","b":true},{"text":": identifiers, code, file paths, quoted material. A summariser will round "},{"text":"ORD-88213","code":true},{"text":" to “the order” and the specificity is gone for good."}],[{"text":"The last few turns","b":true},{"text":", because the immediate thread of the conversation is what the next reply continues."}]]},{"t":"p","runs":[{"text":"Compress.","b":true},{"text":" The content mattered and the wording did not: long explanations the user accepted, tool results already acted on, an earlier draft that has been superseded, a document read three turns ago."}]},{"t":"p","runs":[{"text":"Drop entirely.","b":true},{"text":" Greetings, acknowledgements, “thanks, that’s helpful”, the model’s own restatements of the question, and the apology it produced when it got something wrong. This is more of a transcript than people expect: often 10 to 20% of it."}]},{"t":"p","runs":[{"text":"Never keep.","b":true},{"text":" Anything the user asked you to forget, and anything that should not have been retained under your own retention policy. The forgetting lesson later in this module treats this properly."}]},{"t":"h2","id":"the-shape-that-follows","runs":[{"text":"The shape that follows"}]},{"t":"p","runs":[{"text":"Most systems that handle long conversations well converge on the same layout:"}]},{"t":"code","lang":"","label":"Text","text":"[system prompt]                       pinned, outside the window\n[structured state]                    facts as fields, next lesson\n[first user message]                  pinned verbatim\n[compacted middle]                    a summary of turns 2..n-k\n[last k turns]                        verbatim, k typically 4-8\n[current message]"},{"t":"p","runs":[{"text":"Head and tail verbatim, middle compressed. This is not a coincidence: it is the same U as the position module, arrived at from a different direction. The beginning and the end are where material is used, so those are the parts worth their full token cost."}]},{"t":"h2","id":"choosing-k","runs":[{"text":"Choosing k"}]},{"t":"p","runs":[{"text":"The number of recent turns to keep verbatim is a real parameter and it is worth measuring rather than guessing."}]},{"t":"p","runs":[{"text":"Too small and the model loses the immediate thread: it asks a question that was answered two turns ago, or re-explains something the user already rejected. Too large and the middle gets compacted aggressively to make room, and the conversation loses its longer arc."}]},{"t":"p","runs":[{"text":"Four to eight turns is the usual landing zone. Test it the same way as everything else: take real conversations that went wrong, replay them at k = 2, 4, 8, and see where the failure disappears."}]},{"t":"h2","id":"compress-tool-results-first","runs":[{"text":"Compress tool results first"}]},{"t":"p","runs":[{"text":"If you do only one thing from this lesson, do this. Tool results have the worst value-per-token in the transcript by a wide margin: large, structured, and usually consumed entirely within one turn."}]},{"t":"p","runs":[{"text":"Replace a spent result with a stub:"}]},{"t":"code","lang":"","label":"Text","text":"[tool: search_orders — 8 results for \"late delivery\", used ORD-88213 and\n ORD-88220; full result retained as result_id=r_4471]"},{"t":"p","runs":[{"text":"Thirty tokens instead of twelve thousand, and the "},{"text":"result_id","code":true},{"text":" means the agent can fetch it again if it turns out to be needed. The next module’s lesson on offloading to files generalises this pattern."}]},{"t":"h2","id":"the-audit-that-finds-the-waste","runs":[{"text":"The audit that finds the waste"}]},{"t":"p","runs":[{"text":"Take five long conversations and classify every turn into the four kinds above, with a token count beside each. The proportions are usually startling: acknowledgements and restatements at 15%, spent tool results at 40%, and the constraints that actually govern the task at under 2%."}]},{"t":"p","runs":[{"text":"That single count tells you where the policy should bite, and it almost always says the same thing: the cheapest large win is stubbing tool results, and the most valuable small win is pinning the two per cent that governs everything."}]},{"t":"h2","id":"do-not-delete-the-transcript-you-compacted","runs":[{"text":"Do not delete the transcript you compacted"}]},{"t":"p","runs":[{"text":"Whatever your policy discards from the window, keep in storage. Three reasons: the user may refer to it (“what did you say about the deadline?”), you may need to re-derive a better summary later, and a support investigation into a bad answer needs the raw material rather than your compression of it."}]},{"t":"p","runs":[{"text":"The window is a working set. The transcript is the record. Conflating them means every context decision is also a permanent data-loss decision, which is a much heavier thing to get right."}]}],"check":{"question":[{"text":"A support assistant using a 20-message sliding window starts giving answers unrelated to the customer’s original problem in long conversations. What is the most likely cause?"}],"options":[{"text":[{"text":"Attention dilution means the original message is present but too distant to be used."}],"correct":false,"tempting":[{"text":"A real effect at long context, but it degrades use gradually rather than producing answers unrelated to the problem."}]},{"text":[{"text":"The first message, containing the task, has slid out of the window entirely."}],"correct":true,"tempting":null},{"text":[{"text":"Tool results have accumulated and are crowding out the recent turns."}],"correct":false,"tempting":[{"text":"Would push out recent context rather than the oldest, and the window keeps recent messages by construction."}]},{"text":[{"text":"The model is being consistent with its own earlier errors carried forward in the transcript."}],"correct":false,"tempting":[{"text":"Error carry-forward produces persistently wrong answers about the right problem, not answers about a different one."}]}],"order":[0,2,3,1],"why":null},"prev":{"slug":"a-conversation-is-a-growing-prefix","title":"Every turn carries all the previous ones","minutes":8},"next":{"slug":"compaction-when-and-how","title":"Compacting without losing the thread","minutes":9}}}