{"course":{"slug":"ai-for-students","title":"AI for Students","url":"https://addaly.com/learn/ai-for-students","level":"beginner","track":"work"},"lesson":{"slug":"context-limits-and-forgetting","title":"Context, limits, and why it forgets","url":"https://addaly.com/learn/ai-for-students/context-limits-and-forgetting","position":6,"of":73,"minutes":8,"answer":"Everything the model can use must fit in a fixed token window, it attends least to the middle of a long one, and quotas are measured in the same tokens.","keyIdea":"Everything the model can use must fit in a fixed token window, it attends least to the middle of a long one, and quotas are measured in the same tokens, so pasting less and starting fresh chats improves answers and stretches a free tier at once."},"body":{"format":"markdown","text":"## The window\n\nEverything a model can use to answer you must fit in its **context window**: a fixed number of tokens holding the system instructions, the whole conversation so far, any files you attached, and the answer being written. Nothing outside that window exists for the model.\n\nSizes vary by model and change often. What matters is the arithmetic, not the current number. A 128,000-token window is roughly 90,000 English words, or a 300-page book. An 8,000-token window is roughly a long essay. Your uploaded 60-page PDF might be 30,000 tokens, and if the window is 8,000, most of it is gone before the model reads a word.\n\nWhen the conversation exceeds the window, tools handle it in different ways, and none of them is *remembering*. Some truncate the oldest turns. Some summarise them into a shorter note and drop the originals. Either way, details from early in a long chat stop existing, which is why an assistant that knew your essay's thesis in message four contradicts it in message forty.\n\n## Long context is not the same as attention\n\nA large window means the text *fits*. It does not mean the model uses all of it evenly. Repeated experiments find a pattern researchers describe as *lost in the middle*: information at the very start and the very end of a long context is used reliably, and information buried in the middle is used much less. If you paste a 40-page document and ask about something on page 22, expect worse answers than for page 1 or page 40.\n\n> Figure: Lost in the middle. Line chart of Answered correctly (%) against How far through the pasted text the answer sits (per cent): 40 pages pasted (0, 95) (15, 86) (30, 72) (50, 64) (70, 75) (85, 88) (100, 94); 3 pages pasted (0, 96) (25, 94) (50, 92) (75, 93) (100, 96). One class exercise of sixty questions, in the shape repeated experiments keep finding. A large window means the text fits, not that it is used evenly: material at the very start and the very end is used reliably, and material buried in the middle much less. The fix is not a bigger window. It is pasting the three pages that contain the answer, which gives better replies, faster ones, and a free tier that lasts longer.\n\nThe practical fix is not a bigger window. It is putting less in. Paste the three relevant pages instead of the whole PDF. Quote the paragraph rather than the chapter. You will get better answers and faster ones.\n\n## Why free tiers throttle, in the same units\n\nProviders charge and limit by tokens because tokens are what the computation costs. Every token in the window is processed for every token generated, so a long conversation costs more per reply than a short one: the cost of a chat grows faster than its length.\n\nThis is why:\n\n- A free tier gives you a message allowance that empties faster when your messages are long.\n- Uploading a large PDF can consume a surprising share of a daily quota in one action.\n- The same tool feels more generous in English than in Hindi, Bengali or Tamil, because the same meaning costs more tokens in scripts the tokenizer handles less efficiently.\n\nNone of this requires paying anything to work around. It requires being deliberate about what you put in the window.\n\n## Managing conversations on purpose\n\nThree habits, all free:\n\n1. **Start a new chat when the subject changes.** A fresh window with a clean statement of the problem beats a 60-message thread carrying three abandoned topics. Old context does not help; it distracts, because the model conditions on all of it.\n2. **Re-state the constraints periodically.** In a long working session, every ten or so messages, restate the target: *reminder: 1,200 words, sceptical of the standard reading, A-level not undergraduate*. Cheap, and it survives truncation.\n3. **Keep the source of truth outside the chat.** Your essay lives in a document. Your notes live in a file. The chat is a workspace you can throw away, not a place things are stored. If the conversation is your only copy of something, you will lose it.\n\n## Memory features\n\nSome tools now keep a persistent memory across conversations: your name, your course, your preferences. This is convenient and has two consequences worth knowing. It is stored on the provider's servers, so it is subject to whatever their retention policy says. And it silently conditions future answers, so an assistant that *knows* you are a first-year can quietly simplify things you now need in full. Both are manageable; both are worth checking in the settings rather than assuming.\n\n## Running one on your own machine\n\nIf your laptop has 8GB of RAM you can run a small model locally and free with Ollama or llama.cpp: a 3B or 4B parameter model quantised to about 2-3GB. It is noticeably weaker than the hosted ones and it is entirely private, works offline, and has no quota. For vocabulary drilling, rephrasing, or generating practice questions from text you supply, a small local model is enough. For anything needing broad knowledge, it is not. Knowing which of those you are doing is the skill."},"check":{"question":"A student uploads a 200-page textbook and asks a question about a definition on page 96. The answer is confidently wrong, though the same model answers correctly when given just that page. Why?","options":[{"text":"PDF uploads strip formatting, so the model could not identify which text was a definition.","correct":false,"tempting":"Blames the file format for a retrieval problem; the same PDF page pasted alone produced the right answer."},{"text":"The model's training data did not include that textbook, so it fell back on general knowledge.","correct":false,"tempting":"Confuses training memory with supplied context; supplied text does not need to have been in training to be usable."},{"text":"Material in the middle of a very long context is used far less reliably than material at the start or end.","correct":true},{"text":"The upload exceeded the daily token quota, so the tool silently answered from a truncated summary of the whole book.","correct":false,"tempting":"Conflates billing limits with context handling; a quota breach stops the request rather than degrading the answer invisibly."}]},"prev":{"slug":"the-fluency-trap","title":"The fluency trap","url":"https://addaly.com/learn/ai-for-students/the-fluency-trap"},"next":{"slug":"which-tool-for-which-job","title":"Which tool for which job","url":"https://addaly.com/learn/ai-for-students/which-tool-for-which-job"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Context, limits, and why it forgets', in AI for Students. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-for-students/context-limits-and-forgetting (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Context, limits, and why it forgets","courseTitle":"AI for Students","module":{"index":1,"slug":"what-the-machine-is-doing","title":"What the machine is doing when it helps you"},"answer":null,"keyIdea":"Everything the model can use must fit in a fixed token window, it attends least to the middle of a long one, and quotas are measured in the same tokens, so pasting less and starting fresh chats improves answers and stretches a free tier at once.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"line","title":"Lost in the middle","series":[{"label":"40 pages pasted","points":[[0,95],[15,86],[30,72],[50,64],[70,75],[85,88],[100,94]]},{"label":"3 pages pasted","points":[[0,96],[25,94],[50,92],[75,93],[100,96]]}],"xLabel":"How far through the pasted text the answer sits (per cent)","yLabel":"Answered correctly (%)","caption":"One class exercise of sixty questions, in the shape repeated experiments keep finding. A large window means the text fits, not that it is used evenly: material at the very start and the very end is used reliably, and material buried in the middle much less. The fix is not a bigger window. It is pasting the three pages that contain the answer, which gives better replies, faster ones, and a free tier that lasts longer."},"alt":"Figure: Lost in the middle. Line chart of Answered correctly (%) against How far through the pasted text the answer sits (per cent): 40 pages pasted (0, 95) (15, 86) (30, 72) (50, 64) (70, 75) (85, 88) (100, 94); 3 pages pasted (0, 96) (25, 94) (50, 92) (75, 93) (100, 96). One class exercise of sixty questions, in the shape repeated experiments keep finding. A large window means the text fits, not that it is used evenly: material at the very start and the very end is used reliably, and material buried in the middle much less. The fix is not a bigger window. It is pasting the three pages that contain the answer, which gives better replies, faster ones, and a free tier that lasts longer."}},"blocks":[{"t":"h2","id":"the-window","runs":[{"text":"The window"}]},{"t":"p","runs":[{"text":"Everything a model can use to answer you must fit in its "},{"text":"context window","b":true},{"text":": a fixed number of tokens holding the system instructions, the whole conversation so far, any files you attached, and the answer being written. Nothing outside that window exists for the model."}]},{"t":"p","runs":[{"text":"Sizes vary by model and change often. What matters is the arithmetic, not the current number. A 128,000-token window is roughly 90,000 English words, or a 300-page book. An 8,000-token window is roughly a long essay. Your uploaded 60-page PDF might be 30,000 tokens, and if the window is 8,000, most of it is gone before the model reads a word."}]},{"t":"p","runs":[{"text":"When the conversation exceeds the window, tools handle it in different ways, and none of them is "},{"text":"remembering","i":true},{"text":". Some truncate the oldest turns. Some summarise them into a shorter note and drop the originals. Either way, details from early in a long chat stop existing, which is why an assistant that knew your essay’s thesis in message four contradicts it in message forty."}]},{"t":"h2","id":"long-context-is-not-the-same-as-attention","runs":[{"text":"Long context is not the same as attention"}]},{"t":"p","runs":[{"text":"A large window means the text "},{"text":"fits","i":true},{"text":". It does not mean the model uses all of it evenly. Repeated experiments find a pattern researchers describe as "},{"text":"lost in the middle","i":true},{"text":": information at the very start and the very end of a long context is used reliably, and information buried in the middle is used much less. If you paste a 40-page document and ask about something on page 22, expect worse answers than for page 1 or page 40."}]},{"t":"p","runs":[{"text":"The practical fix is not a bigger window. It is putting less in. Paste the three relevant pages instead of the whole PDF. Quote the paragraph rather than the chapter. You will get better answers and faster ones."}]},{"t":"h2","id":"why-free-tiers-throttle-in-the-same-units","runs":[{"text":"Why free tiers throttle, in the same units"}]},{"t":"p","runs":[{"text":"Providers charge and limit by tokens because tokens are what the computation costs. Every token in the window is processed for every token generated, so a long conversation costs more per reply than a short one: the cost of a chat grows faster than its length."}]},{"t":"p","runs":[{"text":"This is why:"}]},{"t":"ul","items":[[{"text":"A free tier gives you a message allowance that empties faster when your messages are long."}],[{"text":"Uploading a large PDF can consume a surprising share of a daily quota in one action."}],[{"text":"The same tool feels more generous in English than in Hindi, Bengali or Tamil, because the same meaning costs more tokens in scripts the tokenizer handles less efficiently."}]]},{"t":"p","runs":[{"text":"None of this requires paying anything to work around. It requires being deliberate about what you put in the window."}]},{"t":"h2","id":"managing-conversations-on-purpose","runs":[{"text":"Managing conversations on purpose"}]},{"t":"p","runs":[{"text":"Three habits, all free:"}]},{"t":"ol","start":1,"items":[[{"text":"Start a new chat when the subject changes.","b":true},{"text":" A fresh window with a clean statement of the problem beats a 60-message thread carrying three abandoned topics. Old context does not help; it distracts, because the model conditions on all of it."}],[{"text":"Re-state the constraints periodically.","b":true},{"text":" In a long working session, every ten or so messages, restate the target: "},{"text":"reminder: 1,200 words, sceptical of the standard reading, A-level not undergraduate","i":true},{"text":". Cheap, and it survives truncation."}],[{"text":"Keep the source of truth outside the chat.","b":true},{"text":" Your essay lives in a document. Your notes live in a file. The chat is a workspace you can throw away, not a place things are stored. If the conversation is your only copy of something, you will lose it."}]]},{"t":"h2","id":"memory-features","runs":[{"text":"Memory features"}]},{"t":"p","runs":[{"text":"Some tools now keep a persistent memory across conversations: your name, your course, your preferences. This is convenient and has two consequences worth knowing. It is stored on the provider’s servers, so it is subject to whatever their retention policy says. And it silently conditions future answers, so an assistant that "},{"text":"knows","i":true},{"text":" you are a first-year can quietly simplify things you now need in full. Both are manageable; both are worth checking in the settings rather than assuming."}]},{"t":"h2","id":"running-one-on-your-own-machine","runs":[{"text":"Running one on your own machine"}]},{"t":"p","runs":[{"text":"If your laptop has 8GB of RAM you can run a small model locally and free with Ollama or llama.cpp: a 3B or 4B parameter model quantised to about 2-3GB. It is noticeably weaker than the hosted ones and it is entirely private, works offline, and has no quota. For vocabulary drilling, rephrasing, or generating practice questions from text you supply, a small local model is enough. For anything needing broad knowledge, it is not. Knowing which of those you are doing is the skill."}]}],"check":{"question":[{"text":"A student uploads a 200-page textbook and asks a question about a definition on page 96. The answer is confidently wrong, though the same model answers correctly when given just that page. Why?"}],"options":[{"text":[{"text":"PDF uploads strip formatting, so the model could not identify which text was a definition."}],"correct":false,"tempting":[{"text":"Blames the file format for a retrieval problem; the same PDF page pasted alone produced the right answer."}]},{"text":[{"text":"The model’s training data did not include that textbook, so it fell back on general knowledge."}],"correct":false,"tempting":[{"text":"Confuses training memory with supplied context; supplied text does not need to have been in training to be usable."}]},{"text":[{"text":"Material in the middle of a very long context is used far less reliably than material at the start or end."}],"correct":true,"tempting":null},{"text":[{"text":"The upload exceeded the daily token quota, so the tool silently answered from a truncated summary of the whole book."}],"correct":false,"tempting":[{"text":"Conflates billing limits with context handling; a quota breach stops the request rather than degrading the answer invisibly."}]}],"order":[3,2,1,0],"why":null},"prev":{"slug":"the-fluency-trap","title":"The fluency trap","minutes":8},"next":{"slug":"which-tool-for-which-job","title":"Which tool for which job","minutes":9}}}