The six ways a run goes wrong

A vocabulary for what you are looking at

When somebody says “the agent is unreliable”, nothing follows from it. When somebody says “we have tool thrash on the shipping API”, a fix follows immediately. This lesson is the vocabulary.

Six failures cover nearly everything. Each has a mechanism, a symptom in the trace, and a fix.

1. Tool thrash

Symptom. The same tool called with identical arguments three, five, nine times.

Mechanism. The result string carried no information the model could act on: an empty array, Error: 500, a null. Nothing in the context distinguishes “try again” from any other move, so the model repeats its most recent plausible action.

Fix. Informative errors, and a harness-level guard: if a call is byte-identical to one made earlier in the run, return the previous result plus a line saying it is a repeat.

2. Context rot

Symptom. Works for five steps, then re-runs old searches and contradicts findings it established earlier.

Mechanism. The evidence has been pushed out of the window by later observations while the goal remains, so the model reconstructs an approach from the goal.

Fix. Shrink observations at source, compact with the facts preserved, shorten the chain.

3. Fabricated arguments

Symptom. Well-formed identifiers that return 404. ORD-48213 exists; ORD-48214 was invented because it looked like the kind of thing that goes there.

Mechanism. The model produces plausible text. An id-shaped slot gets id-shaped text, and nothing in the model checks a registry.

Fix. A rule in the harness: any identifier in a tool call that did not appear in an earlier tool result is rejected before the call is made, with a message saying so. Enums for anything with a fixed set of values.

4. Silent partial success

Symptom. The final message says the task is complete. Three of the five records were updated.

Mechanism. The model reports on its plan, not on the world. Two calls failed, their errors scrolled past ten steps ago, and the summary was written from intent.

Fix. Completion is checked by code against the world (count the rows that changed) never taken from the model’s own account of itself.

5. Goal drift

Symptom. Asked to reconcile three months of invoices, it produces a careful analysis of one month and stops, satisfied.

Mechanism. The goal was stated once, at the top, and is now eleven thousand tokens away in the least-attended part of the context. The most recent turns are all about January.

Fix. Restate the goal near the end of the context on every step. It costs fifty tokens. Then check the output against the goal in code: three months requested, three months present.

6. The confident wrong finish

Symptom. A clean, well-structured, entirely wrong answer. No errors anywhere in the trace.

Mechanism. Every individual step succeeded. The reasoning connecting them did not, and nothing in the loop was ever asked to check.

Fix. Validators, which are two lessons away, and a human queue for what validators cannot catch.

What they all have in common

Look back at the six symptoms. Not one of them is visible in the final message. The final message is fluent, confident and well-organised in every single case, including the ones where nothing was done.

That is the whole argument for the next lesson. You cannot judge an agent by its output, because its output is the one part it is guaranteed to be good at. You judge it by the trace and by what changed in the world.