{"course":{"slug":"ai-for-small-business","title":"AI for a Small Business","url":"https://addaly.com/learn/ai-for-small-business","level":"beginner","track":"work"},"lesson":{"slug":"context-and-forgetting","title":"Why long conversations go strange","url":"https://addaly.com/learn/ai-for-small-business/context-and-forgetting","position":15,"of":88,"minutes":8,"answer":"The whole thread is re-read on every turn, so long conversations cost more, slow down, and silently drop their oldest material.","keyIdea":"The whole thread is re-read on every turn, so long conversations cost more, slow down, and silently drop their oldest material, which is why standing instructions beat a facts sheet pasted at the top of an hour-old chat."},"body":{"format":"markdown","text":"## Everything is re-read, every time\n\nA chat feels like a conversation with something that remembers. Mechanically it is not. On every turn, the whole thread is sent again (your facts sheet, your examples, all previous questions, all previous answers) and the model reads the lot before producing the next reply.\n\nTwo consequences follow, and both explain things that otherwise look like the tool misbehaving.\n\n**Long threads cost more.** If you pay per use, turn twenty is far more expensive than turn one, because you are paying to re-send everything each time. On a flat subscription you pay in speed instead: replies get slower as the thread grows.\n\n> Figure: What you re-send on every turn. Line chart of tokens sent on that turn against turn in the conversation: Standing instructions of 300 tokens (1, 550) (5, 1550) (10, 2800) (15, 4050) (20, 5300); Standing instructions of 1,500 tokens (1, 1750) (5, 2750) (10, 4000) (15, 5250) (20, 6500). Each turn adds roughly 250 tokens of conversation and the whole thread is read again every time. Turn twenty costs about ten times turn one, which is why the length of your standing instructions is a cost decision as well as a clarity one.\n\n**There is a hard limit.** The **context window** is the maximum amount of text that fits. It is measured in tokens: roughly three quarters of a word. A 128,000-token window is around 96,000 words, or a decent-sized novel. Some models offer a million, some much less.\n\n## What happens at the edge\n\nThis is the part that catches people. When a conversation exceeds the window, most consumer tools do not stop and warn you. They drop the oldest material.\n\nWhich means your facts sheet, pasted at the very top, an hour ago, is the first thing to go. The tool keeps answering, confidently, without it. Nothing announces the change. The replies start being wrong about prices in a way they were not before.\n\nIf you have ever had a long, productive session that inexplicably went downhill, this is usually why.\n\n## Lost in the middle\n\nEven well inside the limit, position matters. Material at the start and the end of a long input is used more reliably than material in the middle. This is a well-documented effect and it is why a model can summarise a long document beautifully and still miss a clause on page 40.\n\nThe practical consequences:\n\n- Put the important instruction **last**, immediately before the request. Not buried in paragraph three of a long preamble.\n- For a long document, ask about **sections** rather than the whole thing.\n- If something must not be missed, say it twice: once at the top, once at the bottom. It is inelegant and it works.\n\n## Thread hygiene\n\nFour habits, all cheap:\n\n**One task, one chat.** Do not answer customers, plan a menu and draft a supplier email in the same thread. The material bleeds, and you pay to re-send all of it.\n\n**Restart at the sign of drift.** If answers get vague, contradict earlier ones, or forget your prices, do not fight it. Restart.\n\n**Hand off deliberately.** Ask for a short summary of decisions so far, paste it into a new chat with your facts sheet, continue. Twenty seconds, and you get back the reliability of a short thread.\n\n**Use standing instructions rather than the top of a thread.** Facts placed in custom instructions or a project are re-supplied every turn by design, rather than sitting at the top of a thread waiting to be dropped. This is the single best defence and it costs nothing.\n\n## Memory features are a different thing\n\nSeveral tools now offer memory that persists across conversations. Useful, and worth understanding as a separate mechanism: the tool is saving notes about you and re-injecting them later.\n\nTwo cautions for a business. It can carry something from a personal chat into a work one, which has produced some awkward drafts. And it is another store of information about your business held by a vendor, which belongs on the one-page tool list in module seven. Check what has been remembered occasionally; most tools let you view and delete it, and most people never look.\n\n## Attachments count too\n\nUploading a file does not put it somewhere separate. In most tools the document's text is read into the same window as everything else, so a 40-page supplier contract consumes a large share of the space your facts sheet and your conversation are sharing.\n\nTwo practical effects. Long documents crowd out your instructions, which is why a request that worked yesterday behaves oddly when you attach a PDF to it. And several large files at once can push you to the edge in a single turn, with no thread history at all.\n\nSome tools handle documents differently, fetching only the relevant passages rather than reading everything. That changes the arithmetic and introduces its own failure, which module three covers. Either way, attach what the task needs rather than everything you have.\n\n## A quick way to tell\n\nIf you suspect the window has been exceeded, ask a question you know the answer to and that only the dropped material could answer. \"What did I say our delivery charge to Ikeja was?\" If the answer is wrong or hedged, the sheet is gone. Restart, rather than continuing to ask a conversation that has quietly forgotten what it was told."},"check":{"question":"An owner pastes her price list at the start of a chat and works through forty enquiries. Around the thirtieth, replies begin quoting prices she does not recognise. What is the most likely mechanism?","options":[{"text":"Accumulated corrections earlier in the thread have taught the model a different set of prices.","correct":false,"tempting":"Plausible where prices were argued over, but corrections would have to have introduced those specific figures, and the drift here begins without any such exchange."},{"text":"The thread has exceeded the context window and the oldest content, including the price list, has been silently dropped.","correct":true},{"text":"The model has switched to a smaller version because of usage limits, and the smaller one is less accurate.","correct":false,"tempting":"Some tools do downgrade under load, but a smaller model given the price list still reads it; the failure here is the list no longer being present."},{"text":"Sampling variation has accumulated over thirty turns, so the outputs have drifted steadily away from the original instructions in the way a photocopy of a photocopy degrades.","correct":false,"tempting":"Treats randomness as cumulative drift, when each turn is generated afresh from whatever is in context and the variation does not compound across turns."}]},"prev":{"slug":"fixing-a-bad-draft","title":"Repairing a draft instead of starting again","url":"https://addaly.com/learn/ai-for-small-business/fixing-a-bad-draft"},"next":{"slug":"the-settings-that-matter","title":"The four settings that change the answer","url":"https://addaly.com/learn/ai-for-small-business/the-settings-that-matter"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Why long conversations go strange', in AI for a Small Business. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-for-small-business/context-and-forgetting (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Why long conversations go strange","courseTitle":"AI for a Small Business","module":{"index":2,"slug":"asking-properly","title":"Asking properly: prompting for business work"},"answer":null,"keyIdea":"The whole thread is re-read on every turn, so long conversations cost more, slow down, and silently drop their oldest material, which is why standing instructions beat a facts sheet pasted at the top of an hour-old chat.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"line","title":"What you re-send on every turn","series":[{"label":"Standing instructions of 300 tokens","points":[[1,550],[5,1550],[10,2800],[15,4050],[20,5300]]},{"label":"Standing instructions of 1,500 tokens","points":[[1,1750],[5,2750],[10,4000],[15,5250],[20,6500]]}],"xLabel":"turn in the conversation","yLabel":"tokens sent on that turn","caption":"Each turn adds roughly 250 tokens of conversation and the whole thread is read again every time. Turn twenty costs about ten times turn one, which is why the length of your standing instructions is a cost decision as well as a clarity one."},"alt":"Figure: What you re-send on every turn. Line chart of tokens sent on that turn against turn in the conversation: Standing instructions of 300 tokens (1, 550) (5, 1550) (10, 2800) (15, 4050) (20, 5300); Standing instructions of 1,500 tokens (1, 1750) (5, 2750) (10, 4000) (15, 5250) (20, 6500). Each turn adds roughly 250 tokens of conversation and the whole thread is read again every time. Turn twenty costs about ten times turn one, which is why the length of your standing instructions is a cost decision as well as a clarity one."}},"blocks":[{"t":"h2","id":"everything-is-re-read-every-time","runs":[{"text":"Everything is re-read, every time"}]},{"t":"p","runs":[{"text":"A chat feels like a conversation with something that remembers. Mechanically it is not. On every turn, the whole thread is sent again (your facts sheet, your examples, all previous questions, all previous answers) and the model reads the lot before producing the next reply."}]},{"t":"p","runs":[{"text":"Two consequences follow, and both explain things that otherwise look like the tool misbehaving."}]},{"t":"p","runs":[{"text":"Long threads cost more.","b":true},{"text":" If you pay per use, turn twenty is far more expensive than turn one, because you are paying to re-send everything each time. On a flat subscription you pay in speed instead: replies get slower as the thread grows."}]},{"t":"p","runs":[{"text":"There is a hard limit.","b":true},{"text":" The "},{"text":"context window","b":true},{"text":" is the maximum amount of text that fits. It is measured in tokens: roughly three quarters of a word. A 128,000-token window is around 96,000 words, or a decent-sized novel. Some models offer a million, some much less."}]},{"t":"h2","id":"what-happens-at-the-edge","runs":[{"text":"What happens at the edge"}]},{"t":"p","runs":[{"text":"This is the part that catches people. When a conversation exceeds the window, most consumer tools do not stop and warn you. They drop the oldest material."}]},{"t":"p","runs":[{"text":"Which means your facts sheet, pasted at the very top, an hour ago, is the first thing to go. The tool keeps answering, confidently, without it. Nothing announces the change. The replies start being wrong about prices in a way they were not before."}]},{"t":"p","runs":[{"text":"If you have ever had a long, productive session that inexplicably went downhill, this is usually why."}]},{"t":"h2","id":"lost-in-the-middle","runs":[{"text":"Lost in the middle"}]},{"t":"p","runs":[{"text":"Even well inside the limit, position matters. Material at the start and the end of a long input is used more reliably than material in the middle. This is a well-documented effect and it is why a model can summarise a long document beautifully and still miss a clause on page 40."}]},{"t":"p","runs":[{"text":"The practical consequences:"}]},{"t":"ul","items":[[{"text":"Put the important instruction "},{"text":"last","b":true},{"text":", immediately before the request. Not buried in paragraph three of a long preamble."}],[{"text":"For a long document, ask about "},{"text":"sections","b":true},{"text":" rather than the whole thing."}],[{"text":"If something must not be missed, say it twice: once at the top, once at the bottom. It is inelegant and it works."}]]},{"t":"h2","id":"thread-hygiene","runs":[{"text":"Thread hygiene"}]},{"t":"p","runs":[{"text":"Four habits, all cheap:"}]},{"t":"p","runs":[{"text":"One task, one chat.","b":true},{"text":" Do not answer customers, plan a menu and draft a supplier email in the same thread. The material bleeds, and you pay to re-send all of it."}]},{"t":"p","runs":[{"text":"Restart at the sign of drift.","b":true},{"text":" If answers get vague, contradict earlier ones, or forget your prices, do not fight it. Restart."}]},{"t":"p","runs":[{"text":"Hand off deliberately.","b":true},{"text":" Ask for a short summary of decisions so far, paste it into a new chat with your facts sheet, continue. Twenty seconds, and you get back the reliability of a short thread."}]},{"t":"p","runs":[{"text":"Use standing instructions rather than the top of a thread.","b":true},{"text":" Facts placed in custom instructions or a project are re-supplied every turn by design, rather than sitting at the top of a thread waiting to be dropped. This is the single best defence and it costs nothing."}]},{"t":"h2","id":"memory-features-are-a-different-thing","runs":[{"text":"Memory features are a different thing"}]},{"t":"p","runs":[{"text":"Several tools now offer memory that persists across conversations. Useful, and worth understanding as a separate mechanism: the tool is saving notes about you and re-injecting them later."}]},{"t":"p","runs":[{"text":"Two cautions for a business. It can carry something from a personal chat into a work one, which has produced some awkward drafts. And it is another store of information about your business held by a vendor, which belongs on the one-page tool list in module seven. Check what has been remembered occasionally; most tools let you view and delete it, and most people never look."}]},{"t":"h2","id":"attachments-count-too","runs":[{"text":"Attachments count too"}]},{"t":"p","runs":[{"text":"Uploading a file does not put it somewhere separate. In most tools the document’s text is read into the same window as everything else, so a 40-page supplier contract consumes a large share of the space your facts sheet and your conversation are sharing."}]},{"t":"p","runs":[{"text":"Two practical effects. Long documents crowd out your instructions, which is why a request that worked yesterday behaves oddly when you attach a PDF to it. And several large files at once can push you to the edge in a single turn, with no thread history at all."}]},{"t":"p","runs":[{"text":"Some tools handle documents differently, fetching only the relevant passages rather than reading everything. That changes the arithmetic and introduces its own failure, which module three covers. Either way, attach what the task needs rather than everything you have."}]},{"t":"h2","id":"a-quick-way-to-tell","runs":[{"text":"A quick way to tell"}]},{"t":"p","runs":[{"text":"If you suspect the window has been exceeded, ask a question you know the answer to and that only the dropped material could answer. “What did I say our delivery charge to Ikeja was?” If the answer is wrong or hedged, the sheet is gone. Restart, rather than continuing to ask a conversation that has quietly forgotten what it was told."}]}],"check":{"question":[{"text":"An owner pastes her price list at the start of a chat and works through forty enquiries. Around the thirtieth, replies begin quoting prices she does not recognise. What is the most likely mechanism?"}],"options":[{"text":[{"text":"Accumulated corrections earlier in the thread have taught the model a different set of prices."}],"correct":false,"tempting":[{"text":"Plausible where prices were argued over, but corrections would have to have introduced those specific figures, and the drift here begins without any such exchange."}]},{"text":[{"text":"The thread has exceeded the context window and the oldest content, including the price list, has been silently dropped."}],"correct":true,"tempting":null},{"text":[{"text":"The model has switched to a smaller version because of usage limits, and the smaller one is less accurate."}],"correct":false,"tempting":[{"text":"Some tools do downgrade under load, but a smaller model given the price list still reads it; the failure here is the list no longer being present."}]},{"text":[{"text":"Sampling variation has accumulated over thirty turns, so the outputs have drifted steadily away from the original instructions in the way a photocopy of a photocopy degrades."}],"correct":false,"tempting":[{"text":"Treats randomness as cumulative drift, when each turn is generated afresh from whatever is in context and the variation does not compound across turns."}]}],"order":[0,1,3,2],"why":null},"prev":{"slug":"fixing-a-bad-draft","title":"Repairing a draft instead of starting again","minutes":8},"next":{"slug":"the-settings-that-matter","title":"The four settings that change the answer","minutes":8}}}