The context window is a ceiling, not a promise

What the number on the box means

A context window is the total amount of text the model can have in front of it at once: your instructions, the documents you attached, the whole conversation so far, and the answer it is currently writing. It is measured in tokens, and a token is roughly three quarters of an English word. Advertised windows currently range from 8,000 tokens to a million and beyond.

What the number guarantees is that the request will not be rejected for being too long. That is all it guarantees. It says nothing about whether the model will use page 180 as well as it uses page 2.

The failure you can reproduce in ten minutes

The pattern has a name: lost in the middle. Models attend more reliably to the beginning and end of a long input than to the middle. Put a crucial sentence 60% of the way through a long document and recall of it drops measurably, on nearly every model, at every price.

Run the test yourself. It takes ten minutes and it will change how you use long documents.

1. Take a long document you have: 50 pages is plenty.
2. Insert one invented sentence about a quarter of the way in: ”The site inspection was carried out by Fatima Nkemdirim on 3 March.”
3. Insert a different invented sentence about two-thirds of the way in.
4. Attach the whole thing and ask two separate questions, one about each sentence.

Do that on three tools. You will get different results, and the differences will be larger than any benchmark gap you have read about. Some tools find both instantly. Some find the first and miss the second. Some tell you the document does not mention it at all, with complete confidence.

Why two products with the same model differ

This is the layer question from the first lesson, in its most expensive form. When you attach a 300-page PDF, the product does one of three things, and it rarely tells you which:

• Sends the whole thing. Best quality, highest cost, only possible if it fits the window.
• Truncates. Sends the first N pages and drops the rest. You get a fluent answer about the first third of your document and no warning.
• Retrieves. Splits the file into chunks, searches for the chunks that look relevant to your question, and sends only those. Excellent for “what does it say about indemnity”, poor for “summarise the whole thing” or “is anything inconsistent”, because the model never sees the whole thing.

A product that retrieves will fail your inserted-sentence test in a specific way: it finds a fact you can search for by keyword and misses one you can only find by reading. That is the tell.

The cost that surprises people

In a chat, the whole conversation is re-sent on every turn. Turn 30 of a long conversation costs far more than turn 1, because everything before it is going up the wire again.

50-page document ~= 30,000 tokens
input price      ~= $3 per million tokens
one question              -> $0.09
twenty questions in
the same conversation     -> the document is re-sent each time
                          -> roughly $1.80, not $0.09

Prompt caching, where the provider stores the processed document and charges a tenth for re-reads, exists on most major APIs and is the fix. In a consumer chat app you have no such control, which is one reason free tiers cap long conversations.

What to do instead of trusting the window

Put the question at the end. Document first, instruction last. This costs nothing and measurably improves recall on long inputs across models.

Ask for locations. “Quote the sentence and give the page number.” A model that cannot produce the quote usually did not have the page.

Split deliberately. Twenty focused questions over five chunks beat one question over 300 pages, and often cost less.

Start a new conversation. When a chat has drifted, the context is full of earlier confusion that is still being re-read every turn. A fresh conversation with a clean summary pasted in is faster, cheaper and usually better.

Do not confuse the window with memory. A window is what fits in one request. Memory across conversations is a product feature built by storing summaries and retrieving them later: a different mechanism with different failure modes, and it is one of the biggest reasons two products running the same model feel different to live with.