{"course":{"slug":"building-with-ai","title":"Building with AI","url":"https://addaly.com/learn/building-with-ai","level":"intermediate","track":"build"},"lesson":{"slug":"what-an-agent-is","title":"What an agent actually is, and why the demos fail","url":"https://addaly.com/learn/building-with-ai/what-an-agent-is","position":26,"of":80,"minutes":8,"answer":"An agent is a loop with a step budget; reliability multiplies, so short chains beat clever ones.","keyIdea":"An agent is a loop with a step budget; reliability multiplies, so short chains beat clever ones."},"body":{"format":"markdown","text":"## The definition, without the marketing\n\nAn agent is a loop. The model gets tools, gets a goal, and on each turn decides which tool to call next. It also decides when to stop. That is the whole thing.\n\n```python\nmessages = [{\"role\": \"user\", \"content\": goal}]\nfor step in range(MAX_STEPS):\n    r = model(messages, tools)\n    messages.append(assistant(r))\n    if r.stop_reason != \"tool_use\":\n        return r                      # it decided it was finished\n    messages.append(run_tools(r, session=session))\nraise StepBudgetExceeded(goal, step)  # loud, not a shrug\n```\n\nCompare that with a pipeline, where *you* decide the steps: extract, then look up, then draft, then send. Same model, same tools. The only difference is who chooses the next move.\n\nThat difference is everything, and it is why the demo is beautiful and the production version is a support ticket.\n\n## Reliability multiplies\n\nSuppose each step is right 95% of the time. That sounds strong. Over a 20-step run, if the steps depend on each other, you get 0.95 to the power of 20, which is about 0.36. The agent completes correctly roughly one time in three.\n\nThis is the single most important fact about agents, and it explains most of what teams observe:\n\n- The demo worked because the demo was four steps.\n- A better model takes each step from 95% to 97%, which moves 20 steps from 36% to 54%. Real, but not the difference between broken and shippable.\n- Cutting the chain from 20 steps to 6 takes 95% per step to 74% overall, without touching the model.\n\nShorter chains beat cleverer ones. Every step you can move out of the loop and into deterministic code is a step that cannot fail.\n\n## Why failures are hard to see\n\nA long run produces a fluent summary at the end: \"I checked the three invoices, reconciled them against the ledger, and flagged one discrepancy.\" It reads like success. Steps 4 through 7 may have returned nothing useful, and the model wrote around the gap, because writing around gaps is what language models are good at.\n\nThree specific things go wrong and hide:\n\n**Silent tool failures.** A tool that returns `{\"status\": \"ok\", \"results\": []}` on an error teaches the model that there is nothing to find. Make failures explicit and make them loud.\n\n> Figure: Why the demo worked and the production run did not. Line chart of Runs that complete correctly, per cent against Steps in the run: 95% per step (1, 95) (4, 81) (6, 74) (10, 60) (15, 46) (20, 36); 97% per step (1, 97) (4, 89) (6, 83) (10, 74) (15, 63) (20, 54). Steps that depend on each other multiply. Upgrading the model from 95 to 97 per cent per step moves a twenty-step run from 36 to 54 per cent, real, and not the difference between broken and shippable. Cutting the same run to six steps reaches 74 per cent without touching the model.\n\n**No ground truth mid-run.** The model cannot tell that step 4 failed. It only sees the text it got back.\n\n**Loops.** Same tool, same arguments, ten times. Detect repeats and break, rather than paying for the same mistake in a circle.\n\n## What actually ships\n\nThe useful production systems people call agents are mostly short, constrained loops:\n\n- **A hard step budget.** Four to eight. When it runs out, raise, do not summarise.\n- **A cost budget too.** Track tokens across the run and stop at a ceiling.\n- **Few tools, sharply separated.** Twenty tools with overlapping descriptions is a routing problem you gave yourself.\n- **Deterministic checkpoints.** After the drafting step, run a real check in code: does the invoice id exist, does the total match, is the recipient in this user's contacts. Do not ask the model whether it did well.\n- **Reversible, idempotent actions.** Draft rather than send. Stage rather than commit. Where an action is irreversible, put a person in front of it.\n- **A plan first.** Ask for the plan, check it in code or with a human, then execute the approved steps. This turns one twenty-step gamble into a checkable decision and a shorter run.\n\nNone of this is exciting, and all of it is the difference between something you can leave running and something you have to watch.\n\n## When not to use a loop\n\nIf you can write down the steps, write down the steps. Autonomy is worth its unreliability only when the sequence varies with the input and you cannot enumerate the cases. Most features people build as agents are pipelines that were never given the chance to be pipelines."},"check":{"question":"An agent that files expense reports works in every demo and fails about half the time in real use. Instrumentation shows each individual step is correct roughly 19 times out of 20. The team's plan is to move to a stronger model. Why should they expect only a modest improvement?","options":[{"text":"The failures look like prompt bugs when examined one at a time, so better prompts would take each step close to perfect.","correct":false,"tempting":"Every individual failure, inspected alone, really does look like something a sharper prompt would have prevented."},{"text":"The transcript grows long over a run and the model loses track, so a larger context window is the real constraint.","correct":false,"tempting":"Long runs do produce long transcripts, and window size is a visible, purchasable knob that feels like the bottleneck."},{"text":"Per-step reliability multiplies across a long chain, so a run fails often even when every step is nearly right; shortening the chain and checking between steps moves the number more than a better model.","correct":true},{"text":"The steps are independent, so the end-to-end failure rate should match the per-step rate; a 50% failure rate means something else is broken.","correct":false,"tempting":"It feels intuitive that a 5% error rate stays 5% however many times you roll, which makes the observed number look like evidence of a separate bug."}]},"prev":{"slug":"designing-a-tool-surface","title":"Designing tools a model can use correctly","url":"https://addaly.com/learn/building-with-ai/designing-a-tool-surface"},"next":{"slug":"writing-the-agent-loop","title":"Writing the loop, with the stopping conditions","url":"https://addaly.com/learn/building-with-ai/writing-the-agent-loop"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What an agent actually is, and why the demos fail', in Building with AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/building-with-ai/what-an-agent-is (Accessed: 31 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What an agent actually is, and why the demos fail","courseTitle":"Building with AI","module":{"index":3,"slug":"structure-tools-and-the-loop","title":"Structure, tools and the loop"},"answer":null,"keyIdea":"An agent is a loop with a step budget; reliability multiplies, so short chains beat clever ones.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"line","title":"Why the demo worked and the production run did not","series":[{"label":"95% per step","points":[[1,95],[4,81],[6,74],[10,60],[15,46],[20,36]]},{"label":"97% per step","points":[[1,97],[4,89],[6,83],[10,74],[15,63],[20,54]]}],"xLabel":"Steps in the run","yLabel":"Runs that complete correctly, per cent","caption":"Steps that depend on each other multiply. Upgrading the model from 95 to 97 per cent per step moves a twenty-step run from 36 to 54 per cent, real, and not the difference between broken and shippable. Cutting the same run to six steps reaches 74 per cent without touching the model."},"alt":"Figure: Why the demo worked and the production run did not. Line chart of Runs that complete correctly, per cent against Steps in the run: 95% per step (1, 95) (4, 81) (6, 74) (10, 60) (15, 46) (20, 36); 97% per step (1, 97) (4, 89) (6, 83) (10, 74) (15, 63) (20, 54). Steps that depend on each other multiply. Upgrading the model from 95 to 97 per cent per step moves a twenty-step run from 36 to 54 per cent, real, and not the difference between broken and shippable. Cutting the same run to six steps reaches 74 per cent without touching the model."}},"blocks":[{"t":"h2","id":"the-definition-without-the-marketing","runs":[{"text":"The definition, without the marketing"}]},{"t":"p","runs":[{"text":"An agent is a loop. The model gets tools, gets a goal, and on each turn decides which tool to call next. It also decides when to stop. That is the whole thing."}]},{"t":"code","lang":"python","label":"Python","text":"messages = [{\"role\": \"user\", \"content\": goal}]\nfor step in range(MAX_STEPS):\n    r = model(messages, tools)\n    messages.append(assistant(r))\n    if r.stop_reason != \"tool_use\":\n        return r                      # it decided it was finished\n    messages.append(run_tools(r, session=session))\nraise StepBudgetExceeded(goal, step)  # loud, not a shrug"},{"t":"p","runs":[{"text":"Compare that with a pipeline, where "},{"text":"you","i":true},{"text":" decide the steps: extract, then look up, then draft, then send. Same model, same tools. The only difference is who chooses the next move."}]},{"t":"p","runs":[{"text":"That difference is everything, and it is why the demo is beautiful and the production version is a support ticket."}]},{"t":"h2","id":"reliability-multiplies","runs":[{"text":"Reliability multiplies"}]},{"t":"p","runs":[{"text":"Suppose each step is right 95% of the time. That sounds strong. Over a 20-step run, if the steps depend on each other, you get 0.95 to the power of 20, which is about 0.36. The agent completes correctly roughly one time in three."}]},{"t":"p","runs":[{"text":"This is the single most important fact about agents, and it explains most of what teams observe:"}]},{"t":"ul","items":[[{"text":"The demo worked because the demo was four steps."}],[{"text":"A better model takes each step from 95% to 97%, which moves 20 steps from 36% to 54%. Real, but not the difference between broken and shippable."}],[{"text":"Cutting the chain from 20 steps to 6 takes 95% per step to 74% overall, without touching the model."}]]},{"t":"p","runs":[{"text":"Shorter chains beat cleverer ones. Every step you can move out of the loop and into deterministic code is a step that cannot fail."}]},{"t":"h2","id":"why-failures-are-hard-to-see","runs":[{"text":"Why failures are hard to see"}]},{"t":"p","runs":[{"text":"A long run produces a fluent summary at the end: “I checked the three invoices, reconciled them against the ledger, and flagged one discrepancy.” It reads like success. Steps 4 through 7 may have returned nothing useful, and the model wrote around the gap, because writing around gaps is what language models are good at."}]},{"t":"p","runs":[{"text":"Three specific things go wrong and hide:"}]},{"t":"p","runs":[{"text":"Silent tool failures.","b":true},{"text":" A tool that returns "},{"text":"{\"status\": \"ok\", \"results\": []}","code":true},{"text":" on an error teaches the model that there is nothing to find. Make failures explicit and make them loud."}]},{"t":"p","runs":[{"text":"No ground truth mid-run.","b":true},{"text":" The model cannot tell that step 4 failed. It only sees the text it got back."}]},{"t":"p","runs":[{"text":"Loops.","b":true},{"text":" Same tool, same arguments, ten times. Detect repeats and break, rather than paying for the same mistake in a circle."}]},{"t":"h2","id":"what-actually-ships","runs":[{"text":"What actually ships"}]},{"t":"p","runs":[{"text":"The useful production systems people call agents are mostly short, constrained loops:"}]},{"t":"ul","items":[[{"text":"A hard step budget.","b":true},{"text":" Four to eight. When it runs out, raise, do not summarise."}],[{"text":"A cost budget too.","b":true},{"text":" Track tokens across the run and stop at a ceiling."}],[{"text":"Few tools, sharply separated.","b":true},{"text":" Twenty tools with overlapping descriptions is a routing problem you gave yourself."}],[{"text":"Deterministic checkpoints.","b":true},{"text":" After the drafting step, run a real check in code: does the invoice id exist, does the total match, is the recipient in this user’s contacts. Do not ask the model whether it did well."}],[{"text":"Reversible, idempotent actions.","b":true},{"text":" Draft rather than send. Stage rather than commit. Where an action is irreversible, put a person in front of it."}],[{"text":"A plan first.","b":true},{"text":" Ask for the plan, check it in code or with a human, then execute the approved steps. This turns one twenty-step gamble into a checkable decision and a shorter run."}]]},{"t":"p","runs":[{"text":"None of this is exciting, and all of it is the difference between something you can leave running and something you have to watch."}]},{"t":"h2","id":"when-not-to-use-a-loop","runs":[{"text":"When not to use a loop"}]},{"t":"p","runs":[{"text":"If you can write down the steps, write down the steps. Autonomy is worth its unreliability only when the sequence varies with the input and you cannot enumerate the cases. Most features people build as agents are pipelines that were never given the chance to be pipelines."}]}],"check":{"question":[{"text":"An agent that files expense reports works in every demo and fails about half the time in real use. Instrumentation shows each individual step is correct roughly 19 times out of 20. The team’s plan is to move to a stronger model. Why should they expect only a modest improvement?"}],"options":[{"text":[{"text":"The failures look like prompt bugs when examined one at a time, so better prompts would take each step close to perfect."}],"correct":false,"tempting":[{"text":"Every individual failure, inspected alone, really does look like something a sharper prompt would have prevented."}]},{"text":[{"text":"The transcript grows long over a run and the model loses track, so a larger context window is the real constraint."}],"correct":false,"tempting":[{"text":"Long runs do produce long transcripts, and window size is a visible, purchasable knob that feels like the bottleneck."}]},{"text":[{"text":"Per-step reliability multiplies across a long chain, so a run fails often even when every step is nearly right; shortening the chain and checking between steps moves the number more than a better model."}],"correct":true,"tempting":null},{"text":[{"text":"The steps are independent, so the end-to-end failure rate should match the per-step rate; a 50% failure rate means something else is broken."}],"correct":false,"tempting":[{"text":"It feels intuitive that a 5% error rate stays 5% however many times you roll, which makes the observed number look like evidence of a separate bug."}]}],"order":[1,0,2,3],"why":null},"prev":{"slug":"designing-a-tool-surface","title":"Designing tools a model can use correctly","minutes":9},"next":{"slug":"writing-the-agent-loop","title":"Writing the loop, with the stopping conditions","minutes":10}}}