The context window is not memory

Every turn sends the whole conversation again

This is what actually leaves your machine on the third message of a chat:

messages = [
  {"role": "user",      "content": "Suggest a name for a tea shop in Lagos."},
  {"role": "assistant", "content": "..."},
  {"role": "user",      "content": "Make it shorter."},
]

All of it. The model holds no session. It has no record that the earlier call happened. "Make it shorter" is meaningful only because the earlier turns are physically present in this request, as tokens, alongside it.

The same is true inside the chat apps. They are assembling this array for you.

The window is a length limit, not a store

The context window — 8,000 tokens, 200,000, a million, depending on the model — is the maximum size of that combined input plus the output being generated. It bounds how much text can be looked at in one pass, the way the size of a desk bounds how many papers you can spread out at once. When the response finishes, nothing is kept.

Three things people get wrong follow directly.

Corrections do not persist. You tell the model your company spells its name with a lowercase letter. It apologises and complies for the rest of the conversation, because your correction is now in the prompt. Tomorrow, in a new chat, it is gone. No weights changed. Nothing was learned.

Everything competes for the same budget. A 100,000-token contract in the prompt is not free background material. It costs the same as 100,000 tokens of conversation, and it eats the room needed for a long answer.

Cost grows faster than length. Attention compares every position with every earlier one. Providers cache the computed keys and values for a prefix they have seen before, which is why re-sending an unchanged system prompt is cheap the second time — and why editing one character near the beginning of a long prompt throws that saving away, since everything after the edit must be recomputed. Put the stable material first and the varying material last.

What "memory" features actually are

When a product remembers that you are vegetarian or that your team uses rupees, a system outside the model wrote that down in a database and injected it into the prompt before the model ever saw your message. Retrieval systems do the same thing with documents: search first, paste the top results into the prompt, then ask.

This is good news, mostly. Such memory can be listed, edited and deleted, and it fails in ordinary database ways rather than mysterious ones. It also means the model is only ever as informed as the retrieval step that fed it, and a retrieval step that returns the wrong paragraph produces a confident answer about the wrong paragraph.

And note what memory is not: fine-tuning. Training on your documents is a different operation with different results, covered in lesson eight.

A long window is not a uniform one

A model can accept a million tokens. That does not mean it uses all of them equally.

Research on long contexts found a consistent pattern nicknamed "lost in the middle": a fact placed at the start or the end of a long prompt is retrieved reliably, while the same fact in the middle is retrieved noticeably less often. Models have improved on the simple version of this test, and the effect varies by model. It remains real for harder cases — reasoning that has to combine several facts scattered through a long document, rather than finding one.

Practical habits that follow. Put your question after the document, not before it. Repeat the key instruction at the end. When a task depends on twenty scattered details, retrieve those twenty passages and send a short prompt rather than sending everything and hoping.