The source or a summary of it

Compression is a decision, not an optimisation

At some point every context pipeline meets material that will not fit: a 40-page contract, an hour of transcript, nine months of conversation. The choice is to include part of it verbatim, or to include a summary of all of it. Both are lossy. They lose different things, and knowing which is which decides the call.

Verbatim excerpt loses whatever you did not select. What survives is exact: every number, name, qualifier and negation as written. The failure is a missing section.

Summary loses precision everywhere and keeps coverage. Summaries systematically drop qualifiers (“usually”, “except where”), collapse specific numbers to approximations, lose the distinction between what a document asserts and what it quotes, and (the one that causes real damage) turn conditional statements into unconditional ones. “Refunds are available within 30 days for unopened items” becomes “refunds are available within 30 days”.

The summary is also model output, which means it is a guess, and the next stage will treat it as a source.

The rule that holds up

Never summarise the text the answer must be grounded in. Summarise the material that provides orientation.

So: summarise conversation history, previous tool results, background documents, the earlier part of a long-running task. Do not summarise the contract clause the question is about, the code the model must modify, the medical note, the passage a citation will point to.

A useful test: if a user might reasonably ask “where exactly does it say that”, the text must be present verbatim. Anything else is fair to compress.

Two places to spend the compute

Summarising costs a model call, which costs money and time. Where you put that call changes the economics entirely.

At ingestion. Summarise each document once, store it, reuse it for every query. Marginal cost at request time is zero. This is where document-level summaries, section abstracts and generated chunk context belong. It is also where you can afford a larger model, since you pay once.

At request time. Summarise only what could not have been anticipated: this conversation’s history, this tool’s oversized result. Adds latency to every call, so use a small fast model and cap the input.

The most common design mistake is doing at request time what could have been done at ingestion. Summarising the same document on every query is paying for the same work thousands of times, and adding a second or two of latency to each one.

The pattern that gets most of both

Summary for orientation, source for grounding, with a path between them:

1. Include a short summary of each candidate document: one or two sentences, generated at ingestion.
2. Include the full text only of the two or three passages the retriever ranked highest.
3. Give the model a tool to fetch the full text of any document by its handle.

The model sees the shape of everything available, reads exactly what it needs, and can go and get more. This costs one extra round trip when the model asks for more, and it turns a fixed budget into an adaptive one. It is also the design that scales best when the corpus is far larger than any window.

Losing the negation

Worth its own warning, because it is the failure that reaches customers.

Summarisers drop negations and exceptions at a measurably higher rate than they drop positive claims. The summary of a page listing four conditions under which something is not permitted often reads as a description of the thing being permitted. Every subsequent stage then works from a document that says the opposite of the original.

Two defences. Where a summary will be used as a source, prompt for the structure rather than for prose (“list every condition and exception, one per line, using the document’s own words for numbers and dates”) which suppresses the fluent-paraphrase behaviour that loses qualifiers. And spot-check summaries against sources on a sample: not a formal evaluation, just twenty documents read side by side. Nearly everyone who does this finds at least one reversal.

Keep the pointer

Whatever you compress, store the link back:

{"summary": "...", "source_id": "doc_8831", "span": [1420, 3980],
 "summarised_by": "haiku-2026-04", "at": "2026-09-01T10:22Z"}

Three things become possible: a user can be shown the original, an evaluation can check the summary against it, and when you eventually discover the summariser had a systematic flaw, you can identify and regenerate everything it produced. A summary with no pointer to its source is a claim with no evidence, and six months later that is exactly what it will be treated as.