{"course":{"slug":"agents-and-automation","title":"Agents and Automation","url":"https://addaly.com/learn/agents-and-automation","level":"intermediate","track":"build"},"lesson":{"slug":"how-agents-fail","title":"The six ways a run goes wrong","url":"https://addaly.com/learn/agents-and-automation/how-agents-fail","position":29,"of":72,"minutes":9,"answer":"Every agent failure mode is invisible in the final message, which is why the final message is the last thing you should judge a run by.","keyIdea":"Every agent failure mode is invisible in the final message, which is why the final message is the last thing you should judge a run by."},"body":{"format":"markdown","text":"## A vocabulary for what you are looking at\n\nWhen somebody says \"the agent is unreliable\", nothing follows from it. When somebody says \"we have tool thrash on the shipping API\", a fix follows immediately. This lesson is the vocabulary.\n\nSix failures cover nearly everything. Each has a mechanism, a symptom in the trace, and a fix.\n\n> Figure: Six failures, and the mechanism under each. Layers, top to bottom: Tool thrash (The same call with the same arguments, nine times. The result string carried nothing to act on.); Context rot (Fine for five steps, then re-runs old searches. The evidence was pushed out while the goal stayed.); Fabricated arguments (Well-formed identifiers that return 404. An id-shaped slot gets id-shaped text.); Silent partial success (It reports five records updated and three were. It summarised its plan, not the world.); Goal drift (Three months requested; one month carefully analysed, and it stops satisfied.); The confident wrong finish (Every individual step succeeded. Nothing in the loop was ever asked to check the reasoning between them.). Not one of the six is visible in the final message, which is fluent, confident and well organised in every case, including the ones where nothing was done. Judge the trace and the state of the world.\n\n## 1. Tool thrash\n\n**Symptom.** The same tool called with identical arguments three, five, nine times.\n\n**Mechanism.** The result string carried no information the model could act on: an empty array, `Error: 500`, a null. Nothing in the context distinguishes \"try again\" from any other move, so the model repeats its most recent plausible action.\n\n**Fix.** Informative errors, and a harness-level guard: if a call is byte-identical to one made earlier in the run, return the previous result plus a line saying it is a repeat.\n\n## 2. Context rot\n\n**Symptom.** Works for five steps, then re-runs old searches and contradicts findings it established earlier.\n\n**Mechanism.** The evidence has been pushed out of the window by later observations while the goal remains, so the model reconstructs an approach from the goal.\n\n**Fix.** Shrink observations at source, compact with the facts preserved, shorten the chain.\n\n## 3. Fabricated arguments\n\n**Symptom.** Well-formed identifiers that return 404. `ORD-48213` exists; `ORD-48214` was invented because it looked like the kind of thing that goes there.\n\n**Mechanism.** The model produces plausible text. An id-shaped slot gets id-shaped text, and nothing in the model checks a registry.\n\n**Fix.** A rule in the harness: **any identifier in a tool call that did not appear in an earlier tool result is rejected before the call is made**, with a message saying so. Enums for anything with a fixed set of values.\n\n## 4. Silent partial success\n\n**Symptom.** The final message says the task is complete. Three of the five records were updated.\n\n**Mechanism.** The model reports on its plan, not on the world. Two calls failed, their errors scrolled past ten steps ago, and the summary was written from intent.\n\n**Fix.** Completion is checked by code against the world (count the rows that changed) never taken from the model's own account of itself.\n\n## 5. Goal drift\n\n**Symptom.** Asked to reconcile three months of invoices, it produces a careful analysis of one month and stops, satisfied.\n\n**Mechanism.** The goal was stated once, at the top, and is now eleven thousand tokens away in the least-attended part of the context. The most recent turns are all about January.\n\n**Fix.** Restate the goal near the end of the context on every step. It costs fifty tokens. Then check the output against the goal in code: three months requested, three months present.\n\n## 6. The confident wrong finish\n\n**Symptom.** A clean, well-structured, entirely wrong answer. No errors anywhere in the trace.\n\n**Mechanism.** Every individual step succeeded. The reasoning connecting them did not, and nothing in the loop was ever asked to check.\n\n**Fix.** Validators, which are two lessons away, and a human queue for what validators cannot catch.\n\n## What they all have in common\n\nLook back at the six symptoms. Not one of them is visible in the final message. The final message is fluent, confident and well-organised in every single case, including the ones where nothing was done.\n\nThat is the whole argument for the next lesson. You cannot judge an agent by its output, because its output is the one part it is guaranteed to be good at. You judge it by the trace and by what changed in the world."},"check":{"question":"An agent is asked to update the shipping address on five orders. It reports 'All five orders updated successfully.' In fact two failed with a validation error at step four, and the trace shows no error after step six. Which failure mode is this, and what is the fix?","options":[{"text":"Silent partial success: the model summarised its plan rather than the world, so completion must be verified in code against the records themselves.","correct":true},{"text":"Fabricated arguments: the model invented order ids that did not exist.","correct":false,"tempting":"Would produce not-found errors on those calls; here the calls were real and two of them were rejected for a different reason."},{"text":"Tool thrash: the failed calls were retried until the loop gave up.","correct":false,"tempting":"Describes repetition, but the trace shows the agent moved on rather than repeating, which is precisely why the failures vanished."},{"text":"Goal drift: the agent switched to an easier version of the task.","correct":false,"tempting":"Names a real mode but the wrong one; the agent attempted all five, so the goal was intact and only the reporting was false."}]},"prev":{"slug":"writing-back-safely","title":"Writing into a system somebody depends on","url":"https://addaly.com/learn/agents-and-automation/writing-back-safely"},"next":{"slug":"observability-and-tracing","title":"You cannot debug what you did not record","url":"https://addaly.com/learn/agents-and-automation/observability-and-tracing"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The six ways a run goes wrong', in Agents and Automation. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/agents-and-automation/how-agents-fail (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The six ways a run goes wrong","courseTitle":"Agents and Automation","module":{"index":4,"slug":"making-it-reliable","title":"Surviving real data"},"answer":null,"keyIdea":"Every agent failure mode is invisible in the final message, which is why the final message is the last thing you should judge a run by.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure 4.1","figure":{"type":"stack","title":"Six failures, and the mechanism under each","layers":[{"label":"Tool thrash","detail":"The same call with the same arguments, nine times. The result string carried nothing to act on."},{"label":"Context rot","detail":"Fine for five steps, then re-runs old searches. The evidence was pushed out while the goal stayed."},{"label":"Fabricated arguments","detail":"Well-formed identifiers that return 404. An id-shaped slot gets id-shaped text."},{"label":"Silent partial success","detail":"It reports five records updated and three were. It summarised its plan, not the world."},{"label":"Goal drift","detail":"Three months requested; one month carefully analysed, and it stops satisfied."},{"label":"The confident wrong finish","detail":"Every individual step succeeded. Nothing in the loop was ever asked to check the reasoning between them."}],"caption":"Not one of the six is visible in the final message, which is fluent, confident and well organised in every case, including the ones where nothing was done. Judge the trace and the state of the world."},"alt":"Figure: Six failures, and the mechanism under each. Layers, top to bottom: Tool thrash (The same call with the same arguments, nine times. The result string carried nothing to act on.); Context rot (Fine for five steps, then re-runs old searches. The evidence was pushed out while the goal stayed.); Fabricated arguments (Well-formed identifiers that return 404. An id-shaped slot gets id-shaped text.); Silent partial success (It reports five records updated and three were. It summarised its plan, not the world.); Goal drift (Three months requested; one month carefully analysed, and it stops satisfied.); The confident wrong finish (Every individual step succeeded. Nothing in the loop was ever asked to check the reasoning between them.). Not one of the six is visible in the final message, which is fluent, confident and well organised in every case, including the ones where nothing was done. Judge the trace and the state of the world."}},"blocks":[{"t":"h2","id":"a-vocabulary-for-what-you-are-looking-at","runs":[{"text":"A vocabulary for what you are looking at"}]},{"t":"p","runs":[{"text":"When somebody says “the agent is unreliable”, nothing follows from it. When somebody says “we have tool thrash on the shipping API”, a fix follows immediately. This lesson is the vocabulary."}]},{"t":"p","runs":[{"text":"Six failures cover nearly everything. Each has a mechanism, a symptom in the trace, and a fix."}]},{"t":"h2","id":"1-tool-thrash","runs":[{"text":"1. Tool thrash"}]},{"t":"p","runs":[{"text":"Symptom.","b":true},{"text":" The same tool called with identical arguments three, five, nine times."}]},{"t":"p","runs":[{"text":"Mechanism.","b":true},{"text":" The result string carried no information the model could act on: an empty array, "},{"text":"Error: 500","code":true},{"text":", a null. Nothing in the context distinguishes “try again” from any other move, so the model repeats its most recent plausible action."}]},{"t":"p","runs":[{"text":"Fix.","b":true},{"text":" Informative errors, and a harness-level guard: if a call is byte-identical to one made earlier in the run, return the previous result plus a line saying it is a repeat."}]},{"t":"h2","id":"2-context-rot","runs":[{"text":"2. Context rot"}]},{"t":"p","runs":[{"text":"Symptom.","b":true},{"text":" Works for five steps, then re-runs old searches and contradicts findings it established earlier."}]},{"t":"p","runs":[{"text":"Mechanism.","b":true},{"text":" The evidence has been pushed out of the window by later observations while the goal remains, so the model reconstructs an approach from the goal."}]},{"t":"p","runs":[{"text":"Fix.","b":true},{"text":" Shrink observations at source, compact with the facts preserved, shorten the chain."}]},{"t":"h2","id":"3-fabricated-arguments","runs":[{"text":"3. Fabricated arguments"}]},{"t":"p","runs":[{"text":"Symptom.","b":true},{"text":" Well-formed identifiers that return 404. "},{"text":"ORD-48213","code":true},{"text":" exists; "},{"text":"ORD-48214","code":true},{"text":" was invented because it looked like the kind of thing that goes there."}]},{"t":"p","runs":[{"text":"Mechanism.","b":true},{"text":" The model produces plausible text. An id-shaped slot gets id-shaped text, and nothing in the model checks a registry."}]},{"t":"p","runs":[{"text":"Fix.","b":true},{"text":" A rule in the harness: "},{"text":"any identifier in a tool call that did not appear in an earlier tool result is rejected before the call is made","b":true},{"text":", with a message saying so. Enums for anything with a fixed set of values."}]},{"t":"h2","id":"4-silent-partial-success","runs":[{"text":"4. Silent partial success"}]},{"t":"p","runs":[{"text":"Symptom.","b":true},{"text":" The final message says the task is complete. Three of the five records were updated."}]},{"t":"p","runs":[{"text":"Mechanism.","b":true},{"text":" The model reports on its plan, not on the world. Two calls failed, their errors scrolled past ten steps ago, and the summary was written from intent."}]},{"t":"p","runs":[{"text":"Fix.","b":true},{"text":" Completion is checked by code against the world (count the rows that changed) never taken from the model’s own account of itself."}]},{"t":"h2","id":"5-goal-drift","runs":[{"text":"5. Goal drift"}]},{"t":"p","runs":[{"text":"Symptom.","b":true},{"text":" Asked to reconcile three months of invoices, it produces a careful analysis of one month and stops, satisfied."}]},{"t":"p","runs":[{"text":"Mechanism.","b":true},{"text":" The goal was stated once, at the top, and is now eleven thousand tokens away in the least-attended part of the context. The most recent turns are all about January."}]},{"t":"p","runs":[{"text":"Fix.","b":true},{"text":" Restate the goal near the end of the context on every step. It costs fifty tokens. Then check the output against the goal in code: three months requested, three months present."}]},{"t":"h2","id":"6-the-confident-wrong-finish","runs":[{"text":"6. The confident wrong finish"}]},{"t":"p","runs":[{"text":"Symptom.","b":true},{"text":" A clean, well-structured, entirely wrong answer. No errors anywhere in the trace."}]},{"t":"p","runs":[{"text":"Mechanism.","b":true},{"text":" Every individual step succeeded. The reasoning connecting them did not, and nothing in the loop was ever asked to check."}]},{"t":"p","runs":[{"text":"Fix.","b":true},{"text":" Validators, which are two lessons away, and a human queue for what validators cannot catch."}]},{"t":"h2","id":"what-they-all-have-in-common","runs":[{"text":"What they all have in common"}]},{"t":"p","runs":[{"text":"Look back at the six symptoms. Not one of them is visible in the final message. The final message is fluent, confident and well-organised in every single case, including the ones where nothing was done."}]},{"t":"p","runs":[{"text":"That is the whole argument for the next lesson. You cannot judge an agent by its output, because its output is the one part it is guaranteed to be good at. You judge it by the trace and by what changed in the world."}]}],"check":{"question":[{"text":"An agent is asked to update the shipping address on five orders. It reports ‘All five orders updated successfully.’ In fact two failed with a validation error at step four, and the trace shows no error after step six. Which failure mode is this, and what is the fix?"}],"options":[{"text":[{"text":"Silent partial success: the model summarised its plan rather than the world, so completion must be verified in code against the records themselves."}],"correct":true,"tempting":null},{"text":[{"text":"Fabricated arguments: the model invented order ids that did not exist."}],"correct":false,"tempting":[{"text":"Would produce not-found errors on those calls; here the calls were real and two of them were rejected for a different reason."}]},{"text":[{"text":"Tool thrash: the failed calls were retried until the loop gave up."}],"correct":false,"tempting":[{"text":"Describes repetition, but the trace shows the agent moved on rather than repeating, which is precisely why the failures vanished."}]},{"text":[{"text":"Goal drift: the agent switched to an easier version of the task."}],"correct":false,"tempting":[{"text":"Names a real mode but the wrong one; the agent attempted all five, so the goal was intact and only the reporting was false."}]}],"order":[3,2,1,0],"why":null},"prev":{"slug":"writing-back-safely","title":"Writing into a system somebody depends on","minutes":10},"next":{"slug":"observability-and-tracing","title":"You cannot debug what you did not record","minutes":9}}}