Context, limits, and why it forgets

The window

Everything a model can use to answer you must fit in its context window: a fixed number of tokens holding the system instructions, the whole conversation so far, any files you attached, and the answer being written. Nothing outside that window exists for the model.

Sizes vary by model and change often. What matters is the arithmetic, not the current number. A 128,000-token window is roughly 90,000 English words, or a 300-page book. An 8,000-token window is roughly a long essay. Your uploaded 60-page PDF might be 30,000 tokens, and if the window is 8,000, most of it is gone before the model reads a word.

When the conversation exceeds the window, tools handle it in different ways, and none of them is remembering. Some truncate the oldest turns. Some summarise them into a shorter note and drop the originals. Either way, details from early in a long chat stop existing, which is why an assistant that knew your essay’s thesis in message four contradicts it in message forty.

Long context is not the same as attention

A large window means the text fits. It does not mean the model uses all of it evenly. Repeated experiments find a pattern researchers describe as lost in the middle: information at the very start and the very end of a long context is used reliably, and information buried in the middle is used much less. If you paste a 40-page document and ask about something on page 22, expect worse answers than for page 1 or page 40.

The practical fix is not a bigger window. It is putting less in. Paste the three relevant pages instead of the whole PDF. Quote the paragraph rather than the chapter. You will get better answers and faster ones.

Why free tiers throttle, in the same units

Providers charge and limit by tokens because tokens are what the computation costs. Every token in the window is processed for every token generated, so a long conversation costs more per reply than a short one: the cost of a chat grows faster than its length.

This is why:

• A free tier gives you a message allowance that empties faster when your messages are long.
• Uploading a large PDF can consume a surprising share of a daily quota in one action.
• The same tool feels more generous in English than in Hindi, Bengali or Tamil, because the same meaning costs more tokens in scripts the tokenizer handles less efficiently.

None of this requires paying anything to work around. It requires being deliberate about what you put in the window.

Managing conversations on purpose

Three habits, all free:

1. Start a new chat when the subject changes. A fresh window with a clean statement of the problem beats a 60-message thread carrying three abandoned topics. Old context does not help; it distracts, because the model conditions on all of it.
2. Re-state the constraints periodically. In a long working session, every ten or so messages, restate the target: reminder: 1,200 words, sceptical of the standard reading, A-level not undergraduate. Cheap, and it survives truncation.
3. Keep the source of truth outside the chat. Your essay lives in a document. Your notes live in a file. The chat is a workspace you can throw away, not a place things are stored. If the conversation is your only copy of something, you will lose it.

Memory features

Some tools now keep a persistent memory across conversations: your name, your course, your preferences. This is convenient and has two consequences worth knowing. It is stored on the provider’s servers, so it is subject to whatever their retention policy says. And it silently conditions future answers, so an assistant that knows you are a first-year can quietly simplify things you now need in full. Both are manageable; both are worth checking in the settings rather than assuming.

Running one on your own machine

If your laptop has 8GB of RAM you can run a small model locally and free with Ollama or llama.cpp: a 3B or 4B parameter model quantised to about 2-3GB. It is noticeably weaker than the hosted ones and it is entirely private, works offline, and has no quota. For vocabulary drilling, rephrasing, or generating practice questions from text you supply, a small local model is enough. For anything needing broad knowledge, it is not. Knowing which of those you are doing is the skill.