{"course":{"slug":"context-engineering","title":"Context Engineering","url":"https://addaly.com/learn/context-engineering","level":"advanced","track":"build"},"lesson":{"slug":"contextual-chunk-augmentation","title":"Giving each chunk its own context","url":"https://addaly.com/learn/context-engineering/contextual-chunk-augmentation","position":23,"of":87,"minutes":9,"answer":"A chunk cut from a long document loses the referents that made it meaningful, so restore them at ingestion (a free heading-path header first.","keyIdea":"A chunk cut from a long document loses the referents that made it meaningful, so restore them at ingestion (a free heading-path header first, then a generated situating sentence indexed alongside the chunk in both vector and keyword indexes) and keep the original text as the thing you actually show the model."},"body":{"format":"markdown","text":"## The chunk that means nothing alone\n\nHere is a real chunk, taken verbatim from the middle of a policy document:\n\n> This does not apply where the item was purchased during a promotional period, or where the customer has already received a replacement under clause 4.\n\nNothing in that text says what *this* is, which product line it covers, or which of your four policies clause 4 belongs to. It will not be retrieved for the query it answers, because it shares no distinctive vocabulary with it. And if it is retrieved, the model cannot use it, because the referent is missing.\n\nThis is the dominant retrieval failure in long structured documents and it is not fixed by a better embedding model. The information the chunk needs was in the document and got left behind at the split.\n\n## Three ways to put the context back, in ascending cost\n\n**1. The structural header: free.** Carry the document title and heading path into the chunk at ingestion time.\n\n```\nRefund policy > Sale items > Exceptions\nThis does not apply where the item was purchased during...\n```\n\nEight to fifteen tokens, no model call, and it fixes a large share of the problem. Embed the chunk *with* this header so the header influences retrieval, not only reading. This should be your default before you consider anything else.\n\n**2. The generated context sentence: one small-model call per chunk.** Ask a cheap model to write one or two sentences situating this chunk inside the whole document, and prepend them.\n\n```\nThis passage is from the Acme refunds policy, revised March 2026, in the\nsection on sale items. It lists the exceptions to the 30-day refund window.\nThis does not apply where the item was purchased during...\n```\n\nAnthropic published results for this technique (contextual embeddings plus contextual keyword indexing) reporting roughly a halving of top-20 retrieval failures on their test corpora, and a larger reduction again when combined with a reranker. Treat the exact figures as vendor-reported on vendor-chosen data, and the direction as well supported: independent teams reproduce a clear improvement on documents with heavy internal cross-reference, and a small one on corpora of short standalone pages.\n\n**3. Overlapping windows and parent expansion.** Retrieve on small chunks, return the larger surrounding block. This is the technique the chunking lesson covers, and it composes with the other two rather than replacing them.\n\n## The cost, and how it collapses\n\nGenerating context per chunk sounds expensive: one call per chunk, and a large corpus has millions. The arithmetic changes completely once you use prompt caching.\n\nThe pattern: put the whole document in the prompt once, cache it, then iterate over its chunks asking for a context sentence for each. Every call after the first reads the document from cache at a fraction of the input price. On a 50-page document with 120 chunks you pay full price for the document once and a cached rate 119 times, plus roughly 100 output tokens each.\n\nThat turns a prohibitive cost into a small one: commonly a dollar or two per thousand pages with a cheap model. It is also entirely an ingestion-time cost, so it adds nothing to request latency, and it is idempotent: re-run it only when the document changes.\n\n## What to put in the generated sentence\n\nSpecific beats fluent. Ask for:\n\n- What document this is and, if dated, when.\n- Where in the document this passage sits.\n- What the passage is about, in terms that do not depend on reading it.\n- Any referent the passage uses without defining: what \"this\", \"the product\" or \"clause 4\" refers to.\n\nAsk explicitly for the last one. It is the highest-value item and the one a model omits unless prompted, because a fluent summary naturally reuses the document's own pronouns.\n\nKeep it to two sentences. The context is paid for on every retrieval of that chunk, forever, and a five-sentence preamble on a 300-token chunk is a poor ratio.\n\n## Index it in both places\n\nThe augmented text should go into the vector index *and* the keyword index. Contextual keyword search matters more than people expect: the generated sentence contains the document title and section names, which are exactly the rare terms a user's query often contains and the chunk body does not.\n\nStore the original chunk text separately from the augmented version. What you show the model, cite from and highlight in a user interface should normally be the original: the generated context is a retrieval aid, and it is model output, which means it can be wrong. Sending it to the model as though it were source text puts a guess in the evidence position.\n\n## Measure it before believing it\n\nThe experiment is clean because it does not touch anything else: index the corpus twice, once plain and once augmented, and run the same queries with the same k. Report recall@k for both.\n\nThe result varies enormously by corpus. Long documents with heavy internal reference gain a lot. A knowledge base of short, standalone, well-titled articles gains almost nothing, because each chunk already carries its context in its own words, and in that case the structural header alone was the right answer and you have just saved yourself the ingestion pipeline."},"check":{"question":"A team adds a generated context sentence to every chunk and sees a large recall gain on their contracts corpus but almost none on their FAQ corpus. What explains the difference?","options":[{"text":"FAQ pages are shorter, so the added context is a larger proportion of the chunk and dilutes its embedding.","correct":false,"tempting":"Dilution is real but would show as a decline; the observed result is no change, which points to the context already being present."},{"text":"FAQ entries are already standalone and self-titled, so nothing was missing for the generated sentence to restore.","correct":true},{"text":"Contracts contain rarer vocabulary, so any change to the text has a bigger effect on embeddings.","correct":false,"tempting":"Attributes the gain to vocabulary rarity rather than to the specific referents (clause numbers and pronouns) that the technique supplies."},{"text":"The FAQ index was not rebuilt after augmentation, so the old embeddings were still being searched.","correct":false,"tempting":"A plausible operational slip, but it would produce identical results rather than the slight change described."}]},"prev":{"slug":"chunking-without-losing-meaning","title":"Chunking that does not destroy meaning","url":"https://addaly.com/learn/context-engineering/chunking-without-losing-meaning"},"next":{"slug":"the-query-is-not-the-question","title":"Rewriting the query before you search","url":"https://addaly.com/learn/context-engineering/the-query-is-not-the-question"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Giving each chunk its own context', in Context Engineering. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/context-engineering/contextual-chunk-augmentation (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Giving each chunk its own context","courseTitle":"Context Engineering","module":{"index":3,"slug":"retrieval-as-a-pipeline","title":"Retrieval as a context pipeline"},"answer":null,"keyIdea":"A chunk cut from a long document loses the referents that made it meaningful, so restore them at ingestion (a free heading-path header first, then a generated situating sentence indexed alongside the chunk in both vector and keyword indexes) and keep the original text as the thing you actually show the model.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-chunk-that-means-nothing-alone","runs":[{"text":"The chunk that means nothing alone"}]},{"t":"p","runs":[{"text":"Here is a real chunk, taken verbatim from the middle of a policy document:"}]},{"t":"quote","runs":[{"text":"This does not apply where the item was purchased during a promotional period, or where the customer has already received a replacement under clause 4."}]},{"t":"p","runs":[{"text":"Nothing in that text says what "},{"text":"this","i":true},{"text":" is, which product line it covers, or which of your four policies clause 4 belongs to. It will not be retrieved for the query it answers, because it shares no distinctive vocabulary with it. And if it is retrieved, the model cannot use it, because the referent is missing."}]},{"t":"p","runs":[{"text":"This is the dominant retrieval failure in long structured documents and it is not fixed by a better embedding model. The information the chunk needs was in the document and got left behind at the split."}]},{"t":"h2","id":"three-ways-to-put-the-context-back-in-ascending-cost","runs":[{"text":"Three ways to put the context back, in ascending cost"}]},{"t":"p","runs":[{"text":"1. The structural header: free.","b":true},{"text":" Carry the document title and heading path into the chunk at ingestion time."}]},{"t":"code","lang":"","label":"Text","text":"Refund policy > Sale items > Exceptions\nThis does not apply where the item was purchased during..."},{"t":"p","runs":[{"text":"Eight to fifteen tokens, no model call, and it fixes a large share of the problem. Embed the chunk "},{"text":"with","i":true},{"text":" this header so the header influences retrieval, not only reading. This should be your default before you consider anything else."}]},{"t":"p","runs":[{"text":"2. The generated context sentence: one small-model call per chunk.","b":true},{"text":" Ask a cheap model to write one or two sentences situating this chunk inside the whole document, and prepend them."}]},{"t":"code","lang":"","label":"Text","text":"This passage is from the Acme refunds policy, revised March 2026, in the\nsection on sale items. It lists the exceptions to the 30-day refund window.\nThis does not apply where the item was purchased during..."},{"t":"p","runs":[{"text":"Anthropic published results for this technique (contextual embeddings plus contextual keyword indexing) reporting roughly a halving of top-20 retrieval failures on their test corpora, and a larger reduction again when combined with a reranker. Treat the exact figures as vendor-reported on vendor-chosen data, and the direction as well supported: independent teams reproduce a clear improvement on documents with heavy internal cross-reference, and a small one on corpora of short standalone pages."}]},{"t":"p","runs":[{"text":"3. Overlapping windows and parent expansion.","b":true},{"text":" Retrieve on small chunks, return the larger surrounding block. This is the technique the chunking lesson covers, and it composes with the other two rather than replacing them."}]},{"t":"h2","id":"the-cost-and-how-it-collapses","runs":[{"text":"The cost, and how it collapses"}]},{"t":"p","runs":[{"text":"Generating context per chunk sounds expensive: one call per chunk, and a large corpus has millions. The arithmetic changes completely once you use prompt caching."}]},{"t":"p","runs":[{"text":"The pattern: put the whole document in the prompt once, cache it, then iterate over its chunks asking for a context sentence for each. Every call after the first reads the document from cache at a fraction of the input price. On a 50-page document with 120 chunks you pay full price for the document once and a cached rate 119 times, plus roughly 100 output tokens each."}]},{"t":"p","runs":[{"text":"That turns a prohibitive cost into a small one: commonly a dollar or two per thousand pages with a cheap model. It is also entirely an ingestion-time cost, so it adds nothing to request latency, and it is idempotent: re-run it only when the document changes."}]},{"t":"h2","id":"what-to-put-in-the-generated-sentence","runs":[{"text":"What to put in the generated sentence"}]},{"t":"p","runs":[{"text":"Specific beats fluent. Ask for:"}]},{"t":"ul","items":[[{"text":"What document this is and, if dated, when."}],[{"text":"Where in the document this passage sits."}],[{"text":"What the passage is about, in terms that do not depend on reading it."}],[{"text":"Any referent the passage uses without defining: what “this”, “the product” or “clause 4” refers to."}]]},{"t":"p","runs":[{"text":"Ask explicitly for the last one. It is the highest-value item and the one a model omits unless prompted, because a fluent summary naturally reuses the document’s own pronouns."}]},{"t":"p","runs":[{"text":"Keep it to two sentences. The context is paid for on every retrieval of that chunk, forever, and a five-sentence preamble on a 300-token chunk is a poor ratio."}]},{"t":"h2","id":"index-it-in-both-places","runs":[{"text":"Index it in both places"}]},{"t":"p","runs":[{"text":"The augmented text should go into the vector index "},{"text":"and","i":true},{"text":" the keyword index. Contextual keyword search matters more than people expect: the generated sentence contains the document title and section names, which are exactly the rare terms a user’s query often contains and the chunk body does not."}]},{"t":"p","runs":[{"text":"Store the original chunk text separately from the augmented version. What you show the model, cite from and highlight in a user interface should normally be the original: the generated context is a retrieval aid, and it is model output, which means it can be wrong. Sending it to the model as though it were source text puts a guess in the evidence position."}]},{"t":"h2","id":"measure-it-before-believing-it","runs":[{"text":"Measure it before believing it"}]},{"t":"p","runs":[{"text":"The experiment is clean because it does not touch anything else: index the corpus twice, once plain and once augmented, and run the same queries with the same k. Report recall@k for both."}]},{"t":"p","runs":[{"text":"The result varies enormously by corpus. Long documents with heavy internal reference gain a lot. A knowledge base of short, standalone, well-titled articles gains almost nothing, because each chunk already carries its context in its own words, and in that case the structural header alone was the right answer and you have just saved yourself the ingestion pipeline."}]}],"check":{"question":[{"text":"A team adds a generated context sentence to every chunk and sees a large recall gain on their contracts corpus but almost none on their FAQ corpus. What explains the difference?"}],"options":[{"text":[{"text":"FAQ pages are shorter, so the added context is a larger proportion of the chunk and dilutes its embedding."}],"correct":false,"tempting":[{"text":"Dilution is real but would show as a decline; the observed result is no change, which points to the context already being present."}]},{"text":[{"text":"FAQ entries are already standalone and self-titled, so nothing was missing for the generated sentence to restore."}],"correct":true,"tempting":null},{"text":[{"text":"Contracts contain rarer vocabulary, so any change to the text has a bigger effect on embeddings."}],"correct":false,"tempting":[{"text":"Attributes the gain to vocabulary rarity rather than to the specific referents (clause numbers and pronouns) that the technique supplies."}]},{"text":[{"text":"The FAQ index was not rebuilt after augmentation, so the old embeddings were still being searched."}],"correct":false,"tempting":[{"text":"A plausible operational slip, but it would produce identical results rather than the slight change described."}]}],"order":[0,1,2,3],"why":null},"prev":{"slug":"chunking-without-losing-meaning","title":"Chunking that does not destroy meaning","minutes":9},"next":{"slug":"the-query-is-not-the-question","title":"Rewriting the query before you search","minutes":9}}}