{"course":{"slug":"building-with-ai","title":"Building with AI","url":"https://addaly.com/learn/building-with-ai","level":"intermediate","track":"build"},"lesson":{"slug":"compacting-history","title":"Compacting history: sliding windows, summaries and what gets lost","url":"https://addaly.com/learn/building-with-ai/compacting-history","position":44,"of":80,"minutes":9,"answer":"A summary of dropped turns keeps a long conversation coherent but loses specifics by its nature (numbers, exact wording.","keyIdea":"A summary of dropped turns keeps a long conversation coherent but loses specifics by its nature (numbers, exact wording, negations) so hard constraints and facts the user stated must be extracted into a separate list that is appended and never summarised."},"body":{"format":"markdown","text":"## Why dropping the oldest is not enough\n\nThe previous lesson trimmed from the oldest turn. For a support chat that is fine. For a long working session it is not. At turn three the user said \"the budget is under 20,000 rupees\". By turn thirty that message is gone, the model suggests something at 35,000, and the user, who can see turn three on their own screen, concludes the product does not listen.\n\nThe cost side pushes the same way. A conversation whose turns average 300 tokens sends 15,000 tokens at turn fifty. Summed across the conversation that is roughly 375,000 input tokens for fifty answers: the quadratic growth the agent-loop lesson warned about, now in a chat.\n\n## Sliding window plus running summary\n\nKeep the last N turns verbatim, and replace everything older with a summary placed directly after the system prompt. When the window overflows again, summarise the old summary together with the turns being dropped, so the summary is updated incrementally rather than regenerated from scratch. Each compaction is one model call with a few hundred output tokens, and it happens every few turns, not every turn.\n\nWith a ten-turn window and a 500-token summary, each request sends about 3,500 tokens instead of 15,000 at the far end, and the fifty-turn conversation costs roughly half what it did. With prompt caching on the stable prefix, less again.\n\n## What to ask the summariser for\n\nNot \"summarise this conversation\". A summary written freely drifts towards narrative (\"the user asked about laptops and the assistant suggested some options\") which preserves the story and drops everything a later turn needs. Ask for structure, using the structured-output pattern from the earlier module:\n\n```text\nProduce JSON with: goal (one sentence), constraints (list, verbatim\nwhere stated), decisions_made (list), open_questions (list),\nfacts_user_stated (list of name: value). Do not paraphrase numbers.\n```\n\n## What gets lost, and why\n\nSummarisation is compression, and compression discards what looks least important to the compressor. Specifics are the first casualty: \"under 20,000\" becomes \"discussed budget\"; \"not the blue one\" becomes \"discussed colour\"; a product code becomes \"a product\". Negations and numbers are short and easy to smooth away, and they are exactly what the user cared about.\n\nThe fix is to stop relying on the summary for them. Maintain a separate list of hard facts and constraints (extracted with a small structured call as each user turn arrives) that is only ever appended to, never summarised, and always placed in the context. The summary carries the shape of the conversation; the fact list carries the details. Two structures, because they have different failure modes.\n\n## Pinning\n\nSome messages must never leave the context: the user's original brief, the key facts extracted from a document they uploaded, a correction they made emphatically. Let a message be pinned, in the data model and optionally in the interface, and exempt pinned messages from trimming. Budget for them; three pinned messages of 800 tokens is a fixed 2,400 tokens on every request.\n\n## Retrieval over the conversation itself\n\nFor very long sessions there is a third tool: embed each past turn, and retrieve the ones relevant to the current message, using the retrieval module's pipeline pointed at the conversation instead of a document set. It answers \"what did I say about the delivery address?\" well. It does not help with questions that need the whole arc, and it adds a retrieval step to every turn. Use it when conversations run to hundreds of turns, which is rarer than people expect.\n\n## Tell the user\n\nWhen compaction has happened, say so, briefly: \"I've condensed the earlier part of our conversation; key details are kept.\" The marker in the interface, at the point where verbatim history stops, does the same job visually. The alternative (silently changing what the model can see) produces the \"it forgot\" complaint with no explanation available to anyone.\n\n## Test it, because it fails quietly\n\nNothing errors when compaction loses a fact. Build evaluation cases for it: a constraint stated at turn two, forty turns of unrelated work, and a question at turn forty-two whose correct answer depends on the constraint. Run them on every change to the summary prompt, the window size or the fact extractor. The evaluation module will make this a habit; this lesson is where the cases come from."},"check":{"question":"A team compacts old turns into a free-text summary. Users report that a budget limit they stated early in a long session is ignored later. The summary prompt is already careful. What is the mechanism and the fix?","options":[{"text":"The summary is placed too late in the context; moving it before the system prompt will make the model weight it more.","correct":false,"tempting":"Position changes emphasis slightly, but the number is no longer in the summary at all, so no position recovers it."},{"text":"The window of verbatim turns is too small; widening it to thirty turns will keep the budget in view for the whole of a normal working session.","correct":false,"tempting":"Delays the failure rather than removing it; the constraint still falls out of the window eventually."},{"text":"Summaries compress away specifics such as numbers by nature; keep stated constraints in a separate list that is appended, never summarised.","correct":true},{"text":"The summariser model is too small; a frontier model will preserve the figure.","correct":false,"tempting":"A stronger model writes a better narrative but still summarises, and the loss is a property of summarising, not of model size."}]},"prev":{"slug":"where-the-conversation-lives","title":"Where the conversation lives","url":"https://addaly.com/learn/building-with-ai/where-the-conversation-lives"},"next":{"slug":"memory-across-sessions","title":"Memory across sessions, and what a user must be able to see","url":"https://addaly.com/learn/building-with-ai/memory-across-sessions"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Compacting history: sliding windows, summaries and what gets lost', in Building with AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/building-with-ai/compacting-history (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Compacting history: sliding windows, summaries and what gets lost","courseTitle":"Building with AI","module":{"index":5,"slug":"conversation-memory-and-the-interface","title":"Conversation, memory and the interface"},"answer":null,"keyIdea":"A summary of dropped turns keeps a long conversation coherent but loses specifics by its nature (numbers, exact wording, negations) so hard constraints and facts the user stated must be extracted into a separate list that is appended and never summarised.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"why-dropping-the-oldest-is-not-enough","runs":[{"text":"Why dropping the oldest is not enough"}]},{"t":"p","runs":[{"text":"The previous lesson trimmed from the oldest turn. For a support chat that is fine. For a long working session it is not. At turn three the user said “the budget is under 20,000 rupees”. By turn thirty that message is gone, the model suggests something at 35,000, and the user, who can see turn three on their own screen, concludes the product does not listen."}]},{"t":"p","runs":[{"text":"The cost side pushes the same way. A conversation whose turns average 300 tokens sends 15,000 tokens at turn fifty. Summed across the conversation that is roughly 375,000 input tokens for fifty answers: the quadratic growth the agent-loop lesson warned about, now in a chat."}]},{"t":"h2","id":"sliding-window-plus-running-summary","runs":[{"text":"Sliding window plus running summary"}]},{"t":"p","runs":[{"text":"Keep the last N turns verbatim, and replace everything older with a summary placed directly after the system prompt. When the window overflows again, summarise the old summary together with the turns being dropped, so the summary is updated incrementally rather than regenerated from scratch. Each compaction is one model call with a few hundred output tokens, and it happens every few turns, not every turn."}]},{"t":"p","runs":[{"text":"With a ten-turn window and a 500-token summary, each request sends about 3,500 tokens instead of 15,000 at the far end, and the fifty-turn conversation costs roughly half what it did. With prompt caching on the stable prefix, less again."}]},{"t":"h2","id":"what-to-ask-the-summariser-for","runs":[{"text":"What to ask the summariser for"}]},{"t":"p","runs":[{"text":"Not “summarise this conversation”. A summary written freely drifts towards narrative (“the user asked about laptops and the assistant suggested some options”) which preserves the story and drops everything a later turn needs. Ask for structure, using the structured-output pattern from the earlier module:"}]},{"t":"code","lang":"text","label":"Text","text":"Produce JSON with: goal (one sentence), constraints (list, verbatim\nwhere stated), decisions_made (list), open_questions (list),\nfacts_user_stated (list of name: value). Do not paraphrase numbers."},{"t":"h2","id":"what-gets-lost-and-why","runs":[{"text":"What gets lost, and why"}]},{"t":"p","runs":[{"text":"Summarisation is compression, and compression discards what looks least important to the compressor. Specifics are the first casualty: “under 20,000” becomes “discussed budget”; “not the blue one” becomes “discussed colour”; a product code becomes “a product”. Negations and numbers are short and easy to smooth away, and they are exactly what the user cared about."}]},{"t":"p","runs":[{"text":"The fix is to stop relying on the summary for them. Maintain a separate list of hard facts and constraints (extracted with a small structured call as each user turn arrives) that is only ever appended to, never summarised, and always placed in the context. The summary carries the shape of the conversation; the fact list carries the details. Two structures, because they have different failure modes."}]},{"t":"h2","id":"pinning","runs":[{"text":"Pinning"}]},{"t":"p","runs":[{"text":"Some messages must never leave the context: the user’s original brief, the key facts extracted from a document they uploaded, a correction they made emphatically. Let a message be pinned, in the data model and optionally in the interface, and exempt pinned messages from trimming. Budget for them; three pinned messages of 800 tokens is a fixed 2,400 tokens on every request."}]},{"t":"h2","id":"retrieval-over-the-conversation-itself","runs":[{"text":"Retrieval over the conversation itself"}]},{"t":"p","runs":[{"text":"For very long sessions there is a third tool: embed each past turn, and retrieve the ones relevant to the current message, using the retrieval module’s pipeline pointed at the conversation instead of a document set. It answers “what did I say about the delivery address?” well. It does not help with questions that need the whole arc, and it adds a retrieval step to every turn. Use it when conversations run to hundreds of turns, which is rarer than people expect."}]},{"t":"h2","id":"tell-the-user","runs":[{"text":"Tell the user"}]},{"t":"p","runs":[{"text":"When compaction has happened, say so, briefly: “I’ve condensed the earlier part of our conversation; key details are kept.” The marker in the interface, at the point where verbatim history stops, does the same job visually. The alternative (silently changing what the model can see) produces the “it forgot” complaint with no explanation available to anyone."}]},{"t":"h2","id":"test-it-because-it-fails-quietly","runs":[{"text":"Test it, because it fails quietly"}]},{"t":"p","runs":[{"text":"Nothing errors when compaction loses a fact. Build evaluation cases for it: a constraint stated at turn two, forty turns of unrelated work, and a question at turn forty-two whose correct answer depends on the constraint. Run them on every change to the summary prompt, the window size or the fact extractor. The evaluation module will make this a habit; this lesson is where the cases come from."}]}],"check":{"question":[{"text":"A team compacts old turns into a free-text summary. Users report that a budget limit they stated early in a long session is ignored later. The summary prompt is already careful. What is the mechanism and the fix?"}],"options":[{"text":[{"text":"The summary is placed too late in the context; moving it before the system prompt will make the model weight it more."}],"correct":false,"tempting":[{"text":"Position changes emphasis slightly, but the number is no longer in the summary at all, so no position recovers it."}]},{"text":[{"text":"The window of verbatim turns is too small; widening it to thirty turns will keep the budget in view for the whole of a normal working session."}],"correct":false,"tempting":[{"text":"Delays the failure rather than removing it; the constraint still falls out of the window eventually."}]},{"text":[{"text":"Summaries compress away specifics such as numbers by nature; keep stated constraints in a separate list that is appended, never summarised."}],"correct":true,"tempting":null},{"text":[{"text":"The summariser model is too small; a frontier model will preserve the figure."}],"correct":false,"tempting":[{"text":"A stronger model writes a better narrative but still summarises, and the loss is a property of summarising, not of model size."}]}],"order":[0,2,1,3],"why":null},"prev":{"slug":"where-the-conversation-lives","title":"Where the conversation lives","minutes":9},"next":{"slug":"memory-across-sessions","title":"Memory across sessions, and what a user must be able to see","minutes":10}}}