{"course":{"slug":"context-engineering","title":"Context Engineering","url":"https://addaly.com/learn/context-engineering","level":"advanced","track":"build"},"lesson":{"slug":"chunking-without-losing-meaning","title":"Chunking that does not destroy meaning","url":"https://addaly.com/learn/context-engineering/chunking-without-losing-meaning","position":22,"of":87,"minutes":9,"answer":"Retrieve the unit a human would quote, and carry down the context that made it mean what it means.","keyIdea":"Retrieve the unit a human would quote, and carry down the context that made it mean what it means."},"body":{"format":"markdown","text":"## The default is wrong for a reason worth understanding\n\nSplit at every 512 tokens with 50 tokens of overlap. It is the default in every framework, and it is wrong in a specific way: it treats a document as a string when a document is a structure.\n\nA fixed split cuts tables away from their headers, separates a clause from the section that qualifies it, and orphans every pronoun. The chunk reads fine to a human skimming, which is why the bug is hard to see, and it is missing the exact word that makes it correct.\n\nHere is the failure that costs money. A refund policy opens with \"The following applies to customers outside the European Union.\" Three pages later, chunk 41 reads \"Refunds are processed within 14 days of receipt.\" Retrieved alone, that chunk is a confident, well-formed, wrong answer for a customer in Dublin. No overlap setting reaches three pages up.\n\n## Split on structure, then on size\n\nThe order matters. Find the boundaries the document already has (headings, sections, list items, table rows, code blocks, slide breaks) and only then merge or split to hit a token budget.\n\n```python\ndef chunk(doc, target=700, floor=200):\n    blocks = parse_structure(doc)        # [{level, heading_path, text, kind}]\n    out, buf, path = [], [], None\n    for b in blocks:\n        if b.kind == \"table\":            # never merge a table into prose\n            out += table_rows(b)         # one chunk per row, header attached\n            continue\n        if path and b.heading_path[:2] != path[:2]:\n            out.append(flush(buf, path)); buf, path = [], b.heading_path\n        buf.append(b.text); path = path or b.heading_path\n        if tokens(buf) >= target:\n            out.append(flush(buf, path)); buf = []\n    return [c for c in out if tokens(c) >= floor or c.kind == \"table_row\"]\n\ndef flush(buf, path):\n    return \" > \".join(path) + \"\\n\\n\" + \"\\n\".join(buf)\n```\n\nThe last line is the important one. Every chunk carries its heading path (`Refunds > Outside the EU > Timelines`) so the qualifier travels with the text. This is a two-line change that fixes the Dublin bug.\n\n## Add the context the chunk lost\n\nGo further: before embedding, prepend one or two sentences that situate the chunk in its document. Anthropic published this as \"contextual retrieval\" in 2024 and reported, on their benchmark, that contextual embeddings cut top-20 retrieval failures by about 35%, and by about 49% when combined with contextual BM25. Treat the exact figures as theirs rather than yours, and treat the direction as reliable.\n\n> Figure: Four ways to cut the same policy corpus. Bars (%): Fixed 512-token split 61; Split on structure first 72; Structure plus heading path 79; Plus a generated situating line 86. Recall at ten on one forty-thousand-chunk corpus. The numbers are from one measurement and will not be yours; the ordering is what travels. Carrying the heading path down onto every chunk is two lines of code, and it is what stops a chunk reading that refunds take fourteen days when the section it came from applies only outside the EU.\n\n```\nThis passage is from the 2026 EMEA refund policy, in the section covering\nnon-EU customers. It follows the eligibility criteria.\n\nRefunds are processed within 14 days of receipt...\n```\n\nYou generate that preamble once per chunk with a cheap model, at ingestion. With prompt caching against the full document, the cost is small (well under a dollar per million tokens of corpus on current small models) and it is paid once, not per query.\n\n## Index small, return large\n\nA short passage embeds precisely. A long passage answers completely. You do not have to choose: embed the small unit, return its parent.\n\nIndex each paragraph or each summary. Store a pointer to the enclosing section. At query time, match on the small thing, retrieve the big thing, deduplicate parents so you do not return the same section four times. Frameworks call this parent-document or small-to-big retrieval; it is thirty lines of code without one.\n\n## The cases that break naive splitters\n\n- **Tables.** One chunk per row, with the header row re-attached and the table's caption in the heading path. A table split at row 40 is unreadable to the model and to you.\n- **Code.** Split on function and class boundaries. Attach the file path and the import block. A function without its imports is missing its type information.\n- **Boilerplate.** Strip repeated headers, footers, page numbers and navigation before embedding. Otherwise every chunk in a 400-page manual shares 30 identical tokens and the embeddings drift towards each other.\n- **Scanned documents.** OCR output has no structure to split on. Recover it (heading detection, layout parsing) or accept that this corpus will retrieve badly.\n- **Conversations.** Split on speaker turns and topic shifts, never on token count. Half a question is worse than nothing.\n\n## How to check your work\n\nSample 30 chunks at random and read them cold, without the source document open. For each, ask: could a careful colleague answer a real question from this alone, and would they know what it applies to?\n\nIf the answer is no, no amount of reranking downstream will save you. Retrieval quality is bounded by the quality of the units you built."},"check":{"question":"A policy corpus is chunked at 300 tokens with 60 tokens of overlap. The system confidently gives EU customers a 14-day refund figure that only applies outside the EU: the qualifier appears once, about 900 tokens earlier under a section heading. A engineer proposes raising the overlap to 150 tokens. What should you expect?","options":[{"text":"It fixes the case, because overlapping windows are precisely the mechanism for carrying preceding context into a chunk.","correct":false,"tempting":"Overlap is designed to preserve continuity across a boundary, so it seems like the right lever, but it only spans the gap it is sized for."},{"text":"It fixes the case indirectly, because larger effective chunks improve embedding quality and the correct passage will rank higher.","correct":false,"tempting":"Chunk size does affect ranking, so this connects two real effects, but the missing qualifier is a content problem rather than a ranking one."},{"text":"It does not fix the case; the qualifier is 900 tokens away, so the fix is to attach the section heading path to every chunk at build time.","correct":true},{"text":"It does not fix the case, and the real fix is a reranker that scores the query against full sections rather than chunks.","correct":false,"tempting":"Reranking genuinely improves ordering, and section-level scoring sounds like it restores context, but a reranker cannot add a qualifier that the returned chunk still does not contain."}]},"prev":{"slug":"documents-that-fight-back","title":"Getting text out of documents that resist","url":"https://addaly.com/learn/context-engineering/documents-that-fight-back"},"next":{"slug":"contextual-chunk-augmentation","title":"Giving each chunk its own context","url":"https://addaly.com/learn/context-engineering/contextual-chunk-augmentation"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Chunking that does not destroy meaning', in Context Engineering. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/context-engineering/chunking-without-losing-meaning (Accessed: 31 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Chunking that does not destroy meaning","courseTitle":"Context Engineering","module":{"index":3,"slug":"retrieval-as-a-pipeline","title":"Retrieval as a context pipeline"},"answer":null,"keyIdea":"Retrieve the unit a human would quote, and carry down the context that made it mean what it means.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"bars","unit":"%","title":"Four ways to cut the same policy corpus","series":[{"label":"Fixed 512-token split","value":61},{"label":"Split on structure first","value":72},{"label":"Structure plus heading path","value":79},{"label":"Plus a generated situating line","value":86,"highlight":true}],"caption":"Recall at ten on one forty-thousand-chunk corpus. The numbers are from one measurement and will not be yours; the ordering is what travels. Carrying the heading path down onto every chunk is two lines of code, and it is what stops a chunk reading that refunds take fourteen days when the section it came from applies only outside the EU."},"alt":"Figure: Four ways to cut the same policy corpus. Bars (%): Fixed 512-token split 61; Split on structure first 72; Structure plus heading path 79; Plus a generated situating line 86. Recall at ten on one forty-thousand-chunk corpus. The numbers are from one measurement and will not be yours; the ordering is what travels. Carrying the heading path down onto every chunk is two lines of code, and it is what stops a chunk reading that refunds take fourteen days when the section it came from applies only outside the EU."}},"blocks":[{"t":"h2","id":"the-default-is-wrong-for-a-reason-worth-understanding","runs":[{"text":"The default is wrong for a reason worth understanding"}]},{"t":"p","runs":[{"text":"Split at every 512 tokens with 50 tokens of overlap. It is the default in every framework, and it is wrong in a specific way: it treats a document as a string when a document is a structure."}]},{"t":"p","runs":[{"text":"A fixed split cuts tables away from their headers, separates a clause from the section that qualifies it, and orphans every pronoun. The chunk reads fine to a human skimming, which is why the bug is hard to see, and it is missing the exact word that makes it correct."}]},{"t":"p","runs":[{"text":"Here is the failure that costs money. A refund policy opens with “The following applies to customers outside the European Union.” Three pages later, chunk 41 reads “Refunds are processed within 14 days of receipt.” Retrieved alone, that chunk is a confident, well-formed, wrong answer for a customer in Dublin. No overlap setting reaches three pages up."}]},{"t":"h2","id":"split-on-structure-then-on-size","runs":[{"text":"Split on structure, then on size"}]},{"t":"p","runs":[{"text":"The order matters. Find the boundaries the document already has (headings, sections, list items, table rows, code blocks, slide breaks) and only then merge or split to hit a token budget."}]},{"t":"code","lang":"python","label":"Python","text":"def chunk(doc, target=700, floor=200):\n    blocks = parse_structure(doc)        # [{level, heading_path, text, kind}]\n    out, buf, path = [], [], None\n    for b in blocks:\n        if b.kind == \"table\":            # never merge a table into prose\n            out += table_rows(b)         # one chunk per row, header attached\n            continue\n        if path and b.heading_path[:2] != path[:2]:\n            out.append(flush(buf, path)); buf, path = [], b.heading_path\n        buf.append(b.text); path = path or b.heading_path\n        if tokens(buf) >= target:\n            out.append(flush(buf, path)); buf = []\n    return [c for c in out if tokens(c) >= floor or c.kind == \"table_row\"]\n\ndef flush(buf, path):\n    return \" > \".join(path) + \"\\n\\n\" + \"\\n\".join(buf)"},{"t":"p","runs":[{"text":"The last line is the important one. Every chunk carries its heading path ("},{"text":"Refunds > Outside the EU > Timelines","code":true},{"text":") so the qualifier travels with the text. This is a two-line change that fixes the Dublin bug."}]},{"t":"h2","id":"add-the-context-the-chunk-lost","runs":[{"text":"Add the context the chunk lost"}]},{"t":"p","runs":[{"text":"Go further: before embedding, prepend one or two sentences that situate the chunk in its document. Anthropic published this as “contextual retrieval” in 2024 and reported, on their benchmark, that contextual embeddings cut top-20 retrieval failures by about 35%, and by about 49% when combined with contextual BM25. Treat the exact figures as theirs rather than yours, and treat the direction as reliable."}]},{"t":"code","lang":"","label":"Text","text":"This passage is from the 2026 EMEA refund policy, in the section covering\nnon-EU customers. It follows the eligibility criteria.\n\nRefunds are processed within 14 days of receipt..."},{"t":"p","runs":[{"text":"You generate that preamble once per chunk with a cheap model, at ingestion. With prompt caching against the full document, the cost is small (well under a dollar per million tokens of corpus on current small models) and it is paid once, not per query."}]},{"t":"h2","id":"index-small-return-large","runs":[{"text":"Index small, return large"}]},{"t":"p","runs":[{"text":"A short passage embeds precisely. A long passage answers completely. You do not have to choose: embed the small unit, return its parent."}]},{"t":"p","runs":[{"text":"Index each paragraph or each summary. Store a pointer to the enclosing section. At query time, match on the small thing, retrieve the big thing, deduplicate parents so you do not return the same section four times. Frameworks call this parent-document or small-to-big retrieval; it is thirty lines of code without one."}]},{"t":"h2","id":"the-cases-that-break-naive-splitters","runs":[{"text":"The cases that break naive splitters"}]},{"t":"ul","items":[[{"text":"Tables.","b":true},{"text":" One chunk per row, with the header row re-attached and the table’s caption in the heading path. A table split at row 40 is unreadable to the model and to you."}],[{"text":"Code.","b":true},{"text":" Split on function and class boundaries. Attach the file path and the import block. A function without its imports is missing its type information."}],[{"text":"Boilerplate.","b":true},{"text":" Strip repeated headers, footers, page numbers and navigation before embedding. Otherwise every chunk in a 400-page manual shares 30 identical tokens and the embeddings drift towards each other."}],[{"text":"Scanned documents.","b":true},{"text":" OCR output has no structure to split on. Recover it (heading detection, layout parsing) or accept that this corpus will retrieve badly."}],[{"text":"Conversations.","b":true},{"text":" Split on speaker turns and topic shifts, never on token count. Half a question is worse than nothing."}]]},{"t":"h2","id":"how-to-check-your-work","runs":[{"text":"How to check your work"}]},{"t":"p","runs":[{"text":"Sample 30 chunks at random and read them cold, without the source document open. For each, ask: could a careful colleague answer a real question from this alone, and would they know what it applies to?"}]},{"t":"p","runs":[{"text":"If the answer is no, no amount of reranking downstream will save you. Retrieval quality is bounded by the quality of the units you built."}]}],"check":{"question":[{"text":"A policy corpus is chunked at 300 tokens with 60 tokens of overlap. The system confidently gives EU customers a 14-day refund figure that only applies outside the EU: the qualifier appears once, about 900 tokens earlier under a section heading. A engineer proposes raising the overlap to 150 tokens. What should you expect?"}],"options":[{"text":[{"text":"It fixes the case, because overlapping windows are precisely the mechanism for carrying preceding context into a chunk."}],"correct":false,"tempting":[{"text":"Overlap is designed to preserve continuity across a boundary, so it seems like the right lever, but it only spans the gap it is sized for."}]},{"text":[{"text":"It fixes the case indirectly, because larger effective chunks improve embedding quality and the correct passage will rank higher."}],"correct":false,"tempting":[{"text":"Chunk size does affect ranking, so this connects two real effects, but the missing qualifier is a content problem rather than a ranking one."}]},{"text":[{"text":"It does not fix the case; the qualifier is 900 tokens away, so the fix is to attach the section heading path to every chunk at build time."}],"correct":true,"tempting":null},{"text":[{"text":"It does not fix the case, and the real fix is a reranker that scores the query against full sections rather than chunks."}],"correct":false,"tempting":[{"text":"Reranking genuinely improves ordering, and section-level scoring sounds like it restores context, but a reranker cannot add a qualifier that the returned chunk still does not contain."}]}],"order":[3,0,2,1],"why":null},"prev":{"slug":"documents-that-fight-back","title":"Getting text out of documents that resist","minutes":10},"next":{"slug":"contextual-chunk-augmentation","title":"Giving each chunk its own context","minutes":9}}}