{"course":{"slug":"agents-and-automation","title":"Agents and Automation","url":"https://addaly.com/learn/agents-and-automation","level":"intermediate","track":"build"},"lesson":{"slug":"observations-that-fit","title":"The observation is where runs die","url":"https://addaly.com/learn/agents-and-automation/observations-that-fit","position":17,"of":72,"minutes":8,"answer":"Shrink the observation at its source, and when you do cut, say in the text that you cut: silent truncation makes the model confident about a subset.","keyIdea":"Shrink the observation at its source, and when you do cut, say in the text that you cut: silent truncation makes the model confident about a subset."},"body":{"format":"markdown","text":"## One step can end a run\n\nA `run_sql` tool with no limit returns four thousand rows. At roughly thirty tokens a row that is 120,000 tokens in a single observation. Depending on the model, the run either errors immediately or spends the rest of its life dragging that block through every subsequent request at full price.\n\nThe plan-act-observe loop is trivial. Choosing what goes back in as the observation is the whole job, and it is mostly a question of size.\n\n> Figure: The same ten steps, at four observation sizes. Bars (thousand input tokens for the whole run): 200 tokens: a row count and five example rows 38; 700 tokens: the extracted table 60.5; 2,000 tokens: a trimmed page 119; 6,000 tokens: the page as fetched 299. One tool that returns the page instead of the table costs about eight times the run. An observation is paid for again on every step after the one that produced it.\n\n## Four ways to shrink one, best first\n\n**1. Ask for less at the source.** A `LIMIT 20`, a projection of four columns instead of forty, a date filter. Costs nothing, loses nothing you asked for, and is the option people skip because it means changing the tool rather than the prompt.\n\n**2. Summarise deterministically.** Code, not a model: the row count, the min and max, the distinct values of the status column, the first five rows. For most analytical steps this is *better* than the raw data, because the model was going to compute those numbers itself and get one of them wrong.\n\n**3. Summarise with a cheap model call.** Genuine option for long prose. It costs money, adds latency, and can drop the one clause that mattered. Use it when the content is unstructured and large, and keep the original addressable by id.\n\n**4. Write it to disk and return the path.** Best when a later step will need the detail. `Saved 3,412 rows to /work/orders.csv (412 KB). Columns: id, status, total, created_at.` The agent can then run code against the file instead of reading it.\n\n## Say that you cut\n\nIf an observation is truncated, the text the model reads must say so. This one line prevents a category of confident wrong answers:\n\n```\nShowing rows 1-20 of 3,412. This is a truncated view.\n```\n\nWithout it the model will write \"there are 20 pending orders\" and be entirely reasonable in doing so, because you told it there were.\n\n## Compacting the whole transcript\n\nOnce the transcript passes roughly half the window, replace the middle of it with a summary. What you preserve is the craft:\n\n- **the goal**, verbatim, never summarised\n- **decisions made and why**, one line each\n- **facts discovered**, with the id or path they came from\n- **things already tried that failed**, or the agent will try them again\n- **the last two turns, unchanged**, so the immediate thread is intact\n\nNote that \"already tried and failed\" is the item everyone forgets, and it is the one that prevents the loop.\n\n## The failure this fixes\n\nYou have already met the symptom: an agent that works for five steps, then re-runs searches it has done and contradicts something it established earlier.\n\nThe mechanism is now visible. The evidence has been squeezed out of the window while the goal remains, so the model does the only thing it can: reconstructs an approach from the goal, which is exactly what it did the first time. The agent is not confused. It is looking at a different document from the one you think it is looking at.\n\n## Measure before you send\n\nEvery provider offers a way to count tokens before a request, and `tiktoken` and equivalents run locally. Put a check in the harness: if any single observation exceeds some fraction of the window (five per cent is a reasonable starting point) summarise it before it enters the transcript, and log that you did."},"check":{"question":"A research agent has a `search_docs` tool that returns the full text of the top 10 matches. Runs work for four or five steps and then start repeating earlier searches. Which change most directly addresses the mechanism?","options":[{"text":"Raise the step budget so the agent has room to finish.","correct":false,"tempting":"Gives more steps to a loop that is already going backwards; the extra steps will be spent repeating the same searches at higher cost."},{"text":"Return id, title and a short snippet per match, and add a `read_doc` tool, so full text enters the context only when it is needed.","correct":true},{"text":"Lower the temperature so the model stops choosing different searches.","correct":false,"tempting":"Treats a context problem as a sampling problem; the searches repeat because the earlier evidence is gone, not because sampling is noisy."},{"text":"Switch to a model with a longer context window.","correct":false,"tempting":"Buys time rather than fixing the cause, and attention over very long contexts degrades in the middle exactly where the earlier evidence sits."}]},"prev":{"slug":"designing-a-tool-surface","title":"Tools that survive being called by a machine","url":"https://addaly.com/learn/agents-and-automation/designing-a-tool-surface"},"next":{"slug":"context-window-management","title":"What to throw away, and when","url":"https://addaly.com/learn/agents-and-automation/context-window-management"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The observation is where runs die', in Agents and Automation. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/agents-and-automation/observations-that-fit (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The observation is where runs die","courseTitle":"Agents and Automation","module":{"index":2,"slug":"how-the-loop-runs","title":"The machinery: transcript, tools, loop"},"answer":null,"keyIdea":"Shrink the observation at its source, and when you do cut, say in the text that you cut: silent truncation makes the model confident about a subset.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"bars","unit":"thousand input tokens for the whole run","title":"The same ten steps, at four observation sizes","series":[{"label":"200 tokens: a row count and five example rows","value":38},{"label":"700 tokens: the extracted table","value":60.5},{"label":"2,000 tokens: a trimmed page","value":119},{"label":"6,000 tokens: the page as fetched","value":299,"highlight":true}],"caption":"One tool that returns the page instead of the table costs about eight times the run. An observation is paid for again on every step after the one that produced it."},"alt":"Figure: The same ten steps, at four observation sizes. Bars (thousand input tokens for the whole run): 200 tokens: a row count and five example rows 38; 700 tokens: the extracted table 60.5; 2,000 tokens: a trimmed page 119; 6,000 tokens: the page as fetched 299. One tool that returns the page instead of the table costs about eight times the run. An observation is paid for again on every step after the one that produced it."}},"blocks":[{"t":"h2","id":"one-step-can-end-a-run","runs":[{"text":"One step can end a run"}]},{"t":"p","runs":[{"text":"A "},{"text":"run_sql","code":true},{"text":" tool with no limit returns four thousand rows. At roughly thirty tokens a row that is 120,000 tokens in a single observation. Depending on the model, the run either errors immediately or spends the rest of its life dragging that block through every subsequent request at full price."}]},{"t":"p","runs":[{"text":"The plan-act-observe loop is trivial. Choosing what goes back in as the observation is the whole job, and it is mostly a question of size."}]},{"t":"h2","id":"four-ways-to-shrink-one-best-first","runs":[{"text":"Four ways to shrink one, best first"}]},{"t":"p","runs":[{"text":"1. Ask for less at the source.","b":true},{"text":" A "},{"text":"LIMIT 20","code":true},{"text":", a projection of four columns instead of forty, a date filter. Costs nothing, loses nothing you asked for, and is the option people skip because it means changing the tool rather than the prompt."}]},{"t":"p","runs":[{"text":"2. Summarise deterministically.","b":true},{"text":" Code, not a model: the row count, the min and max, the distinct values of the status column, the first five rows. For most analytical steps this is "},{"text":"better","i":true},{"text":" than the raw data, because the model was going to compute those numbers itself and get one of them wrong."}]},{"t":"p","runs":[{"text":"3. Summarise with a cheap model call.","b":true},{"text":" Genuine option for long prose. It costs money, adds latency, and can drop the one clause that mattered. Use it when the content is unstructured and large, and keep the original addressable by id."}]},{"t":"p","runs":[{"text":"4. Write it to disk and return the path.","b":true},{"text":" Best when a later step will need the detail. "},{"text":"Saved 3,412 rows to /work/orders.csv (412 KB). Columns: id, status, total, created_at.","code":true},{"text":" The agent can then run code against the file instead of reading it."}]},{"t":"h2","id":"say-that-you-cut","runs":[{"text":"Say that you cut"}]},{"t":"p","runs":[{"text":"If an observation is truncated, the text the model reads must say so. This one line prevents a category of confident wrong answers:"}]},{"t":"code","lang":"","label":"Text","text":"Showing rows 1-20 of 3,412. This is a truncated view."},{"t":"p","runs":[{"text":"Without it the model will write “there are 20 pending orders” and be entirely reasonable in doing so, because you told it there were."}]},{"t":"h2","id":"compacting-the-whole-transcript","runs":[{"text":"Compacting the whole transcript"}]},{"t":"p","runs":[{"text":"Once the transcript passes roughly half the window, replace the middle of it with a summary. What you preserve is the craft:"}]},{"t":"ul","items":[[{"text":"the goal","b":true},{"text":", verbatim, never summarised"}],[{"text":"decisions made and why","b":true},{"text":", one line each"}],[{"text":"facts discovered","b":true},{"text":", with the id or path they came from"}],[{"text":"things already tried that failed","b":true},{"text":", or the agent will try them again"}],[{"text":"the last two turns, unchanged","b":true},{"text":", so the immediate thread is intact"}]]},{"t":"p","runs":[{"text":"Note that “already tried and failed” is the item everyone forgets, and it is the one that prevents the loop."}]},{"t":"h2","id":"the-failure-this-fixes","runs":[{"text":"The failure this fixes"}]},{"t":"p","runs":[{"text":"You have already met the symptom: an agent that works for five steps, then re-runs searches it has done and contradicts something it established earlier."}]},{"t":"p","runs":[{"text":"The mechanism is now visible. The evidence has been squeezed out of the window while the goal remains, so the model does the only thing it can: reconstructs an approach from the goal, which is exactly what it did the first time. The agent is not confused. It is looking at a different document from the one you think it is looking at."}]},{"t":"h2","id":"measure-before-you-send","runs":[{"text":"Measure before you send"}]},{"t":"p","runs":[{"text":"Every provider offers a way to count tokens before a request, and "},{"text":"tiktoken","code":true},{"text":" and equivalents run locally. Put a check in the harness: if any single observation exceeds some fraction of the window (five per cent is a reasonable starting point) summarise it before it enters the transcript, and log that you did."}]}],"check":{"question":[{"text":"A research agent has a "},{"text":"search_docs","code":true},{"text":" tool that returns the full text of the top 10 matches. Runs work for four or five steps and then start repeating earlier searches. Which change most directly addresses the mechanism?"}],"options":[{"text":[{"text":"Raise the step budget so the agent has room to finish."}],"correct":false,"tempting":[{"text":"Gives more steps to a loop that is already going backwards; the extra steps will be spent repeating the same searches at higher cost."}]},{"text":[{"text":"Return id, title and a short snippet per match, and add a "},{"text":"read_doc","code":true},{"text":" tool, so full text enters the context only when it is needed."}],"correct":true,"tempting":null},{"text":[{"text":"Lower the temperature so the model stops choosing different searches."}],"correct":false,"tempting":[{"text":"Treats a context problem as a sampling problem; the searches repeat because the earlier evidence is gone, not because sampling is noisy."}]},{"text":[{"text":"Switch to a model with a longer context window."}],"correct":false,"tempting":[{"text":"Buys time rather than fixing the cause, and attention over very long contexts degrades in the middle exactly where the earlier evidence sits."}]}],"order":[0,3,2,1],"why":null},"prev":{"slug":"designing-a-tool-surface","title":"Tools that survive being called by a machine","minutes":9},"next":{"slug":"context-window-management","title":"What to throw away, and when","minutes":10}}}