Why long conversations go strange

Everything is re-read, every time

A chat feels like a conversation with something that remembers. Mechanically it is not. On every turn, the whole thread is sent again (your facts sheet, your examples, all previous questions, all previous answers) and the model reads the lot before producing the next reply.

Two consequences follow, and both explain things that otherwise look like the tool misbehaving.

Long threads cost more. If you pay per use, turn twenty is far more expensive than turn one, because you are paying to re-send everything each time. On a flat subscription you pay in speed instead: replies get slower as the thread grows.

There is a hard limit. The context window is the maximum amount of text that fits. It is measured in tokens: roughly three quarters of a word. A 128,000-token window is around 96,000 words, or a decent-sized novel. Some models offer a million, some much less.

What happens at the edge

This is the part that catches people. When a conversation exceeds the window, most consumer tools do not stop and warn you. They drop the oldest material.

Which means your facts sheet, pasted at the very top, an hour ago, is the first thing to go. The tool keeps answering, confidently, without it. Nothing announces the change. The replies start being wrong about prices in a way they were not before.

If you have ever had a long, productive session that inexplicably went downhill, this is usually why.

Lost in the middle

Even well inside the limit, position matters. Material at the start and the end of a long input is used more reliably than material in the middle. This is a well-documented effect and it is why a model can summarise a long document beautifully and still miss a clause on page 40.

The practical consequences:

• Put the important instruction last, immediately before the request. Not buried in paragraph three of a long preamble.
• For a long document, ask about sections rather than the whole thing.
• If something must not be missed, say it twice: once at the top, once at the bottom. It is inelegant and it works.

Thread hygiene

Four habits, all cheap:

One task, one chat. Do not answer customers, plan a menu and draft a supplier email in the same thread. The material bleeds, and you pay to re-send all of it.

Restart at the sign of drift. If answers get vague, contradict earlier ones, or forget your prices, do not fight it. Restart.

Hand off deliberately. Ask for a short summary of decisions so far, paste it into a new chat with your facts sheet, continue. Twenty seconds, and you get back the reliability of a short thread.

Use standing instructions rather than the top of a thread. Facts placed in custom instructions or a project are re-supplied every turn by design, rather than sitting at the top of a thread waiting to be dropped. This is the single best defence and it costs nothing.

Memory features are a different thing

Several tools now offer memory that persists across conversations. Useful, and worth understanding as a separate mechanism: the tool is saving notes about you and re-injecting them later.

Two cautions for a business. It can carry something from a personal chat into a work one, which has produced some awkward drafts. And it is another store of information about your business held by a vendor, which belongs on the one-page tool list in module seven. Check what has been remembered occasionally; most tools let you view and delete it, and most people never look.

Attachments count too

Uploading a file does not put it somewhere separate. In most tools the document’s text is read into the same window as everything else, so a 40-page supplier contract consumes a large share of the space your facts sheet and your conversation are sharing.

Two practical effects. Long documents crowd out your instructions, which is why a request that worked yesterday behaves oddly when you attach a PDF to it. And several large files at once can push you to the edge in a single turn, with no thread history at all.

Some tools handle documents differently, fetching only the relevant passages rather than reading everything. That changes the arithmetic and introduces its own failure, which module three covers. Either way, attach what the task needs rather than everything you have.

A quick way to tell

If you suspect the window has been exceeded, ask a question you know the answer to and that only the dropped material could answer. “What did I say our delivery charge to Ikeja was?” If the answer is wrong or hedged, the sheet is gone. Restart, rather than continuing to ask a conversation that has quietly forgotten what it was told.