{"course":{"slug":"context-engineering","title":"Context Engineering","url":"https://addaly.com/learn/context-engineering","level":"advanced","track":"build"},"lesson":{"slug":"compaction-when-and-how","title":"Compacting without losing the thread","url":"https://addaly.com/learn/context-engineering/compaction-when-and-how","position":51,"of":87,"minutes":9,"answer":"Trigger compaction on a fraction of the token budget rather than on turn count.","keyIdea":"Trigger compaction on a fraction of the token budget rather than on turn count, ask for structured fields (especially decisions with rejections and approaches already tried) re-derive each compaction from the raw transcript rather than from the previous summary, and check that every id and number in the record appears in the source."},"body":{"format":"markdown","text":"## Trigger on tokens, not on turns\n\nCompact when the history block crosses a fraction of its budget (70% is a reasonable default) not after a fixed number of messages. Turns vary from four tokens to fourteen thousand, so a turn count is a proxy for the thing you actually care about, and a bad one.\n\nCompacting at 70% rather than 100% matters. It leaves room for the next few turns, so you are not compacting on every single message once you reach the ceiling, which would be both expensive and behaviourally jarring.\n\n```python\nif tokens(history) > 0.7 * BUDGET[\"history\"]:\n    head, middle, tail = split(history, keep_first=1, keep_last=K)\n    history = [head] + [compact(middle)] + tail\n```\n\n## Ask for structure, not for prose\n\nThe single biggest quality difference between a good compaction and a bad one is what you ask for. A request to \"summarise the conversation so far\" produces fluent narrative that loses exactly the things you needed: numbers, names, constraints, and what was ruled out.\n\nAsk for fields:\n\n```\nProduce a compact record of this conversation with these headings.\nUse the exact wording for names, numbers, ids and quoted text.\n\nTASK: what the user is trying to achieve, in their words\nCONSTRAINTS: every requirement or restriction stated, one per line\nDECISIONS: what has been settled, and what was rejected and why\nFACTS: specific values established (ids, dates, amounts, names)\nTRIED: approaches attempted and their outcomes\nOPEN: questions still unanswered\n```\n\nTwo of those headings do most of the work. **DECISIONS** including rejections stops the conversation looping back to an option already dismissed, which is the most visible symptom of bad compaction. **TRIED** stops the model re-attempting something that failed: the same reason it is the most valuable line in a handover note.\n\n## Compact from the raw transcript, not from the last summary\n\nA summary of a summary loses more at each step, and the loss is not random: specifics go first, then qualifiers, then the distinction between what was decided and what was merely discussed. After four rounds you have a general description of a conversation that no longer constrains anything.\n\nSince you kept the full transcript, re-derive each compaction from the original messages rather than from the previous summary. It costs more tokens per compaction and it stops the drift. Where a conversation is enormous, compact in segments from raw and concatenate the segment records, rather than recursively compressing a compression.\n\n## Expect a discontinuity, and place it well\n\nCompaction rewrites the middle of the prefix. Two things follow.\n\nThe **cache is invalidated** from the compaction point onwards. You will pay a cache write on the next call. This is a good reason not to compact every turn and a good reason to compact a large chunk when you do.\n\nThe **behaviour visibly changes**. The model has a different, thinner view of the conversation than it did a moment ago, and users notice as a shift in tone or a suddenly repeated question. Where you can, compact at a natural boundary (after a task completes, when the user changes subject, at the start of a session) rather than mid-thread.\n\n## Verify the compaction\n\nIt is model output, so it can be wrong, and it is about to become the only record the model has. Two cheap checks worth running.\n\n**Containment.** Every id, number and date in the record should appear somewhere in the source transcript. A regex over both catches a fabricated figure immediately.\n\n```python\nfor tok in re.findall(r\"\\b[A-Z]{2,}-\\d+|\\b\\d[\\d,\\.]*\\b\", record):\n    assert tok in raw_transcript, f\"invented value: {tok}\"\n```\n\n**Coverage.** Did each constraint the user stated survive? If your structured state (the next lesson) already tracks constraints as fields, this is a set comparison rather than a judgement.\n\n## What compaction cannot do\n\nBe honest about the trade. Compaction is lossy by definition; the only question is which loss you choose. Structured extraction loses the texture and the reasoning; prose summary loses the specifics; dropping loses everything.\n\nThere is no configuration that makes a 200,000-token conversation fit in 6,000 tokens without loss. What you can do is choose to lose the things that will not be needed, keep a pointer to everything, and make the loss visible to the user rather than silent. The last lesson in this module is about that visibility, and it is the part that turns an engineering limitation into an acceptable product."},"check":{"question":"A long-running assistant repeatedly proposes an approach the user rejected an hour earlier. Compaction is running. What is the most direct fix?","options":[{"text":"Compact less often, so more of the conversation stays verbatim.","correct":false,"tempting":"Delays the loss without preventing it; the rejection will still be dropped once compaction eventually runs."},{"text":"Include a decisions field that records what was rejected and why, not only what was chosen.","correct":true},{"text":"Increase the number of recent turns kept verbatim.","correct":false,"tempting":"Helps with the immediate thread but not with a decision made an hour and dozens of turns ago."},{"text":"Compact from the previous summary rather than the raw transcript, to keep the record stable.","correct":false,"tempting":"Recursive compression is precisely what accelerates the loss of specifics like rejected options."}]},"prev":{"slug":"what-to-keep-verbatim","title":"Which turns are worth their tokens","url":"https://addaly.com/learn/context-engineering/what-to-keep-verbatim"},"next":{"slug":"the-facts-that-must-survive","title":"State as fields, not as prose","url":"https://addaly.com/learn/context-engineering/the-facts-that-must-survive"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Compacting without losing the thread', in Context Engineering. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/context-engineering/compaction-when-and-how (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Compacting without losing the thread","courseTitle":"Context Engineering","module":{"index":6,"slug":"conversation-and-memory","title":"Conversation, memory and compaction"},"answer":null,"keyIdea":"Trigger compaction on a fraction of the token budget rather than on turn count, ask for structured fields (especially decisions with rejections and approaches already tried) re-derive each compaction from the raw transcript rather than from the previous summary, and check that every id and number in the record appears in the source.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"trigger-on-tokens-not-on-turns","runs":[{"text":"Trigger on tokens, not on turns"}]},{"t":"p","runs":[{"text":"Compact when the history block crosses a fraction of its budget (70% is a reasonable default) not after a fixed number of messages. Turns vary from four tokens to fourteen thousand, so a turn count is a proxy for the thing you actually care about, and a bad one."}]},{"t":"p","runs":[{"text":"Compacting at 70% rather than 100% matters. It leaves room for the next few turns, so you are not compacting on every single message once you reach the ceiling, which would be both expensive and behaviourally jarring."}]},{"t":"code","lang":"python","label":"Python","text":"if tokens(history) > 0.7 * BUDGET[\"history\"]:\n    head, middle, tail = split(history, keep_first=1, keep_last=K)\n    history = [head] + [compact(middle)] + tail"},{"t":"h2","id":"ask-for-structure-not-for-prose","runs":[{"text":"Ask for structure, not for prose"}]},{"t":"p","runs":[{"text":"The single biggest quality difference between a good compaction and a bad one is what you ask for. A request to “summarise the conversation so far” produces fluent narrative that loses exactly the things you needed: numbers, names, constraints, and what was ruled out."}]},{"t":"p","runs":[{"text":"Ask for fields:"}]},{"t":"code","lang":"","label":"Text","text":"Produce a compact record of this conversation with these headings.\nUse the exact wording for names, numbers, ids and quoted text.\n\nTASK: what the user is trying to achieve, in their words\nCONSTRAINTS: every requirement or restriction stated, one per line\nDECISIONS: what has been settled, and what was rejected and why\nFACTS: specific values established (ids, dates, amounts, names)\nTRIED: approaches attempted and their outcomes\nOPEN: questions still unanswered"},{"t":"p","runs":[{"text":"Two of those headings do most of the work. "},{"text":"DECISIONS","b":true},{"text":" including rejections stops the conversation looping back to an option already dismissed, which is the most visible symptom of bad compaction. "},{"text":"TRIED","b":true},{"text":" stops the model re-attempting something that failed: the same reason it is the most valuable line in a handover note."}]},{"t":"h2","id":"compact-from-the-raw-transcript-not-from-the-last-summary","runs":[{"text":"Compact from the raw transcript, not from the last summary"}]},{"t":"p","runs":[{"text":"A summary of a summary loses more at each step, and the loss is not random: specifics go first, then qualifiers, then the distinction between what was decided and what was merely discussed. After four rounds you have a general description of a conversation that no longer constrains anything."}]},{"t":"p","runs":[{"text":"Since you kept the full transcript, re-derive each compaction from the original messages rather than from the previous summary. It costs more tokens per compaction and it stops the drift. Where a conversation is enormous, compact in segments from raw and concatenate the segment records, rather than recursively compressing a compression."}]},{"t":"h2","id":"expect-a-discontinuity-and-place-it-well","runs":[{"text":"Expect a discontinuity, and place it well"}]},{"t":"p","runs":[{"text":"Compaction rewrites the middle of the prefix. Two things follow."}]},{"t":"p","runs":[{"text":"The "},{"text":"cache is invalidated","b":true},{"text":" from the compaction point onwards. You will pay a cache write on the next call. This is a good reason not to compact every turn and a good reason to compact a large chunk when you do."}]},{"t":"p","runs":[{"text":"The "},{"text":"behaviour visibly changes","b":true},{"text":". The model has a different, thinner view of the conversation than it did a moment ago, and users notice as a shift in tone or a suddenly repeated question. Where you can, compact at a natural boundary (after a task completes, when the user changes subject, at the start of a session) rather than mid-thread."}]},{"t":"h2","id":"verify-the-compaction","runs":[{"text":"Verify the compaction"}]},{"t":"p","runs":[{"text":"It is model output, so it can be wrong, and it is about to become the only record the model has. Two cheap checks worth running."}]},{"t":"p","runs":[{"text":"Containment.","b":true},{"text":" Every id, number and date in the record should appear somewhere in the source transcript. A regex over both catches a fabricated figure immediately."}]},{"t":"code","lang":"python","label":"Python","text":"for tok in re.findall(r\"\\b[A-Z]{2,}-\\d+|\\b\\d[\\d,\\.]*\\b\", record):\n    assert tok in raw_transcript, f\"invented value: {tok}\""},{"t":"p","runs":[{"text":"Coverage.","b":true},{"text":" Did each constraint the user stated survive? If your structured state (the next lesson) already tracks constraints as fields, this is a set comparison rather than a judgement."}]},{"t":"h2","id":"what-compaction-cannot-do","runs":[{"text":"What compaction cannot do"}]},{"t":"p","runs":[{"text":"Be honest about the trade. Compaction is lossy by definition; the only question is which loss you choose. Structured extraction loses the texture and the reasoning; prose summary loses the specifics; dropping loses everything."}]},{"t":"p","runs":[{"text":"There is no configuration that makes a 200,000-token conversation fit in 6,000 tokens without loss. What you can do is choose to lose the things that will not be needed, keep a pointer to everything, and make the loss visible to the user rather than silent. The last lesson in this module is about that visibility, and it is the part that turns an engineering limitation into an acceptable product."}]}],"check":{"question":[{"text":"A long-running assistant repeatedly proposes an approach the user rejected an hour earlier. Compaction is running. What is the most direct fix?"}],"options":[{"text":[{"text":"Compact less often, so more of the conversation stays verbatim."}],"correct":false,"tempting":[{"text":"Delays the loss without preventing it; the rejection will still be dropped once compaction eventually runs."}]},{"text":[{"text":"Include a decisions field that records what was rejected and why, not only what was chosen."}],"correct":true,"tempting":null},{"text":[{"text":"Increase the number of recent turns kept verbatim."}],"correct":false,"tempting":[{"text":"Helps with the immediate thread but not with a decision made an hour and dozens of turns ago."}]},{"text":[{"text":"Compact from the previous summary rather than the raw transcript, to keep the record stable."}],"correct":false,"tempting":[{"text":"Recursive compression is precisely what accelerates the loss of specifics like rejected options."}]}],"order":[3,1,2,0],"why":null},"prev":{"slug":"what-to-keep-verbatim","title":"Which turns are worth their tokens","minutes":8},"next":{"slug":"the-facts-that-must-survive","title":"State as fields, not as prose","minutes":8}}}