{"course":{"slug":"building-with-ai","title":"Building with AI","url":"https://addaly.com/learn/building-with-ai","level":"intermediate","track":"build"},"lesson":{"slug":"decomposition-and-chaining","title":"One big prompt, or four small ones","url":"https://addaly.com/learn/building-with-ai/decomposition-and-chaining","position":13,"of":80,"minutes":9,"answer":"Split a prompt when the pieces need different models, different tests or a checkable intermediate, put a deterministic check between every link, and stop at three or four steps because accuracy multiplies down the chain.","keyIdea":"Split a prompt when the pieces need different models, different tests or a checkable intermediate, put a deterministic check between every link, and stop at three or four steps because accuracy multiplies down the chain."},"body":{"format":"markdown","text":"## The mega-prompt and how it fails\n\nA prompt that asks for six things at once (read this ticket, classify it, check it against policy, draft a reply, decide whether to escalate, and output JSON) fails in a characteristic way. It does not fail on all six. It degrades on two of them, usually the ones in the middle, and because the output still looks complete you do not notice for a month.\n\nThe mechanism is attention and instruction competition. Everything you asked for is competing for the same generation, and the parts that are hardest or least emphasised lose. You also cannot debug it: when the JSON has a wrong `escalate` flag, you have no way to know whether the classification was wrong, the policy lookup was wrong, or only the final decision was wrong.\n\n**Splitting is primarily a debuggability decision.** Four prompts have four inspectable outputs, and each one can have its own test cases, its own model, and its own fix.\n\n## When to split\n\nSplit when any of these is true:\n\n- **Different steps deserve different models.** Classification runs fine on the cheap tier; drafting the customer-facing reply may not. One prompt forces you to pay frontier prices for the classification too.\n- **A step's output is checkable.** If step two produces a category, you can validate it against a fixed list before step three ever runs. Errors caught early do not propagate.\n- **A step can be skipped.** Half of tickets need no policy lookup. In a mega-prompt you pay for the lookup instructions every time.\n- **A step can be cached or reused.** A document summary computed once serves twenty questions.\n- **You need to log an intermediate value** for audit or for evaluation.\n\nKeep it as one prompt when the steps are interdependent: where step two's correct behaviour depends on nuance from step one that a short intermediate representation would destroy. Summarising then translating is worse than translating a summary written with translation in mind, because the intermediate throws information away.\n\n## The routing pattern\n\nThe most valuable chain in ordinary product work is two links: a cheap classifier that decides which specialised prompt handles the request.\n\n```python\nkind = classify(msg)                    # small model, 40 tokens out, ~15ms\nhandler = {\"billing\": billing_prompt,\n           \"technical\": tech_prompt,\n           \"account\": account_prompt}.get(kind, generic_prompt)\nreturn handler(msg, retrieve(kind, msg))\n```\n\nThis wins three times over. Each specialised prompt is short, so it is cheaper and more compliant than one prompt trying to cover every case. Retrieval is scoped to the right corpus, which improves the hit rate more than any amount of prompt tuning. And when billing answers get worse, you edit the billing prompt without any risk to the other three.\n\nThe cost of the extra call is small: a 300-token classification on a cheap model is a fraction of a cent and 15 milliseconds. The cost of *not* routing shows up as a 1,500-token prompt on every request.\n\n> Figure: Four small prompts, or one big one. Split it when: The steps deserve different models; A step's output can be checked in code; Half of requests can skip a step; A step's result is reused across questions; An intermediate value must be logged. Keep it as one prompt when: Step two depends on nuance step one would discard; The intermediate representation throws information away; The chain would run past three or four links; Nobody on the team can hold the diagram in their head. Splitting is a debuggability decision before it is anything else. It is also a cost decision, because a chain lets a cheap model do the cheap steps. Four steps at 95 per cent each is 81 per cent end to end, so a check between every link is what keeps the arithmetic survivable.\n\n## Errors compound, so put the checks between the links\n\nThe arithmetic is unforgiving. Four steps at 95 per cent each give 0.95⁴ ≈ 81 per cent end to end. Six steps at 90 per cent give 53 per cent. This is the same multiplication that makes agent demos fail, and a chain is a fixed agent.\n\nThe defence is that a fixed chain lets you put a deterministic check at each junction, and a check that can reject early stops the error from propagating:\n\n```python\ncat = classify(msg)\nif cat not in CATEGORIES:            # cheap, certain, catches drift\n    cat = \"other\"\nfacts = extract(msg, cat)\nvalidate(facts)                       # schema, ranges, required fields\n```\n\nEach check converts a silent wrong answer into a loud failure you can route or retry. That is the whole value: **you cannot make each step more accurate by wishing, but you can stop a bad step from reaching the next one.**\n\n## Latency, and what to run in parallel\n\nA chain is serial by default, and four 800-millisecond calls is 3.2 seconds. Two structural fixes.\n\n**Fan out where steps are independent.** Classification and PII redaction do not depend on each other; run them concurrently and pay for the slower one only.\n\n**Start the expensive step early.** If retrieval does not depend on classification, fire it in parallel and discard the wrong branch. You paid for a query you did not need; you saved a round trip on every request.\n\nAnd measure before you optimise. It is common for the whole chain to be dominated by one 2-second frontier call, in which case restructuring the cheap steps changes nothing a user can feel.\n\n## The failure mode of chains\n\nOne honest warning. A long chain becomes a system that nobody can hold in their head, where each step is individually reasonable and the whole produces answers no one can explain. Three or four steps is usually the point where the debuggability gain reverses.\n\nIf your diagram needs a diagram, you have not decomposed the problem; you have distributed it."},"check":{"question":"A single prompt does classification, extraction and reply drafting, and the reply is occasionally wrong in a way nobody can diagnose. What is the primary argument for splitting it?","options":[{"text":"Three shorter prompts cost less in total than one long one.","correct":false,"tempting":"Usually false: three calls repeat their own overheads, and the saving comes from routing and model choice rather than from splitting itself."},{"text":"Each stage produces an inspectable, testable output, so a wrong reply can be attributed to a specific stage and fixed there.","correct":true},{"text":"Shorter prompts have higher accuracy per step, so the end-to-end accuracy rises.","correct":false,"tempting":"Ignores that accuracy multiplies across steps; splitting can lower end-to-end accuracy unless checks are added between links."},{"text":"Splitting lets the model think for longer about each part.","correct":false,"tempting":"Treats separate calls as extra deliberation, when each call is an independent generation with no memory of the others' reasoning."}]},"prev":{"slug":"examples-that-teach","title":"Examples that teach, and examples that mislead","url":"https://addaly.com/learn/building-with-ai/examples-that-teach"},"next":{"slug":"thinking-budgets","title":"Reasoning models, thinking budgets, and when they are a waste of money","url":"https://addaly.com/learn/building-with-ai/thinking-budgets"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'One big prompt, or four small ones', in Building with AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/building-with-ai/decomposition-and-chaining (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"One big prompt, or four small ones","courseTitle":"Building with AI","module":{"index":2,"slug":"prompting-as-engineering","title":"Prompting as engineering"},"answer":null,"keyIdea":"Split a prompt when the pieces need different models, different tests or a checkable intermediate, put a deterministic check between every link, and stop at three or four steps because accuracy multiplies down the chain.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"left":{"title":"Split it when","items":["The steps deserve different models","A step’s output can be checked in code","Half of requests can skip a step","A step’s result is reused across questions","An intermediate value must be logged"]},"type":"compare","right":{"title":"Keep it as one prompt when","items":["Step two depends on nuance step one would discard","The intermediate representation throws information away","The chain would run past three or four links","Nobody on the team can hold the diagram in their head"]},"title":"Four small prompts, or one big one","caption":"Splitting is a debuggability decision before it is anything else. It is also a cost decision, because a chain lets a cheap model do the cheap steps. Four steps at 95 per cent each is 81 per cent end to end, so a check between every link is what keeps the arithmetic survivable."},"alt":"Figure: Four small prompts, or one big one. Split it when: The steps deserve different models; A step’s output can be checked in code; Half of requests can skip a step; A step’s result is reused across questions; An intermediate value must be logged. Keep it as one prompt when: Step two depends on nuance step one would discard; The intermediate representation throws information away; The chain would run past three or four links; Nobody on the team can hold the diagram in their head. Splitting is a debuggability decision before it is anything else. It is also a cost decision, because a chain lets a cheap model do the cheap steps. Four steps at 95 per cent each is 81 per cent end to end, so a check between every link is what keeps the arithmetic survivable."}},"blocks":[{"t":"h2","id":"the-mega-prompt-and-how-it-fails","runs":[{"text":"The mega-prompt and how it fails"}]},{"t":"p","runs":[{"text":"A prompt that asks for six things at once (read this ticket, classify it, check it against policy, draft a reply, decide whether to escalate, and output JSON) fails in a characteristic way. It does not fail on all six. It degrades on two of them, usually the ones in the middle, and because the output still looks complete you do not notice for a month."}]},{"t":"p","runs":[{"text":"The mechanism is attention and instruction competition. Everything you asked for is competing for the same generation, and the parts that are hardest or least emphasised lose. You also cannot debug it: when the JSON has a wrong "},{"text":"escalate","code":true},{"text":" flag, you have no way to know whether the classification was wrong, the policy lookup was wrong, or only the final decision was wrong."}]},{"t":"p","runs":[{"text":"Splitting is primarily a debuggability decision.","b":true},{"text":" Four prompts have four inspectable outputs, and each one can have its own test cases, its own model, and its own fix."}]},{"t":"h2","id":"when-to-split","runs":[{"text":"When to split"}]},{"t":"p","runs":[{"text":"Split when any of these is true:"}]},{"t":"ul","items":[[{"text":"Different steps deserve different models.","b":true},{"text":" Classification runs fine on the cheap tier; drafting the customer-facing reply may not. One prompt forces you to pay frontier prices for the classification too."}],[{"text":"A step’s output is checkable.","b":true},{"text":" If step two produces a category, you can validate it against a fixed list before step three ever runs. Errors caught early do not propagate."}],[{"text":"A step can be skipped.","b":true},{"text":" Half of tickets need no policy lookup. In a mega-prompt you pay for the lookup instructions every time."}],[{"text":"A step can be cached or reused.","b":true},{"text":" A document summary computed once serves twenty questions."}],[{"text":"You need to log an intermediate value","b":true},{"text":" for audit or for evaluation."}]]},{"t":"p","runs":[{"text":"Keep it as one prompt when the steps are interdependent: where step two’s correct behaviour depends on nuance from step one that a short intermediate representation would destroy. Summarising then translating is worse than translating a summary written with translation in mind, because the intermediate throws information away."}]},{"t":"h2","id":"the-routing-pattern","runs":[{"text":"The routing pattern"}]},{"t":"p","runs":[{"text":"The most valuable chain in ordinary product work is two links: a cheap classifier that decides which specialised prompt handles the request."}]},{"t":"code","lang":"python","label":"Python","text":"kind = classify(msg)                    # small model, 40 tokens out, ~15ms\nhandler = {\"billing\": billing_prompt,\n           \"technical\": tech_prompt,\n           \"account\": account_prompt}.get(kind, generic_prompt)\nreturn handler(msg, retrieve(kind, msg))"},{"t":"p","runs":[{"text":"This wins three times over. Each specialised prompt is short, so it is cheaper and more compliant than one prompt trying to cover every case. Retrieval is scoped to the right corpus, which improves the hit rate more than any amount of prompt tuning. And when billing answers get worse, you edit the billing prompt without any risk to the other three."}]},{"t":"p","runs":[{"text":"The cost of the extra call is small: a 300-token classification on a cheap model is a fraction of a cent and 15 milliseconds. The cost of "},{"text":"not","i":true},{"text":" routing shows up as a 1,500-token prompt on every request."}]},{"t":"h2","id":"errors-compound-so-put-the-checks-between-the-links","runs":[{"text":"Errors compound, so put the checks between the links"}]},{"t":"p","runs":[{"text":"The arithmetic is unforgiving. Four steps at 95 per cent each give 0.95⁴ ≈ 81 per cent end to end. Six steps at 90 per cent give 53 per cent. This is the same multiplication that makes agent demos fail, and a chain is a fixed agent."}]},{"t":"p","runs":[{"text":"The defence is that a fixed chain lets you put a deterministic check at each junction, and a check that can reject early stops the error from propagating:"}]},{"t":"code","lang":"python","label":"Python","text":"cat = classify(msg)\nif cat not in CATEGORIES:            # cheap, certain, catches drift\n    cat = \"other\"\nfacts = extract(msg, cat)\nvalidate(facts)                       # schema, ranges, required fields"},{"t":"p","runs":[{"text":"Each check converts a silent wrong answer into a loud failure you can route or retry. That is the whole value: "},{"text":"you cannot make each step more accurate by wishing, but you can stop a bad step from reaching the next one.","b":true}]},{"t":"h2","id":"latency-and-what-to-run-in-parallel","runs":[{"text":"Latency, and what to run in parallel"}]},{"t":"p","runs":[{"text":"A chain is serial by default, and four 800-millisecond calls is 3.2 seconds. Two structural fixes."}]},{"t":"p","runs":[{"text":"Fan out where steps are independent.","b":true},{"text":" Classification and PII redaction do not depend on each other; run them concurrently and pay for the slower one only."}]},{"t":"p","runs":[{"text":"Start the expensive step early.","b":true},{"text":" If retrieval does not depend on classification, fire it in parallel and discard the wrong branch. You paid for a query you did not need; you saved a round trip on every request."}]},{"t":"p","runs":[{"text":"And measure before you optimise. It is common for the whole chain to be dominated by one 2-second frontier call, in which case restructuring the cheap steps changes nothing a user can feel."}]},{"t":"h2","id":"the-failure-mode-of-chains","runs":[{"text":"The failure mode of chains"}]},{"t":"p","runs":[{"text":"One honest warning. A long chain becomes a system that nobody can hold in their head, where each step is individually reasonable and the whole produces answers no one can explain. Three or four steps is usually the point where the debuggability gain reverses."}]},{"t":"p","runs":[{"text":"If your diagram needs a diagram, you have not decomposed the problem; you have distributed it."}]}],"check":{"question":[{"text":"A single prompt does classification, extraction and reply drafting, and the reply is occasionally wrong in a way nobody can diagnose. What is the primary argument for splitting it?"}],"options":[{"text":[{"text":"Three shorter prompts cost less in total than one long one."}],"correct":false,"tempting":[{"text":"Usually false: three calls repeat their own overheads, and the saving comes from routing and model choice rather than from splitting itself."}]},{"text":[{"text":"Each stage produces an inspectable, testable output, so a wrong reply can be attributed to a specific stage and fixed there."}],"correct":true,"tempting":null},{"text":[{"text":"Shorter prompts have higher accuracy per step, so the end-to-end accuracy rises."}],"correct":false,"tempting":[{"text":"Ignores that accuracy multiplies across steps; splitting can lower end-to-end accuracy unless checks are added between links."}]},{"text":[{"text":"Splitting lets the model think for longer about each part."}],"correct":false,"tempting":[{"text":"Treats separate calls as extra deliberation, when each call is an independent generation with no memory of the others’ reasoning."}]}],"order":[3,0,2,1],"why":null},"prev":{"slug":"examples-that-teach","title":"Examples that teach, and examples that mislead","minutes":9},"next":{"slug":"thinking-budgets","title":"Reasoning models, thinking budgets, and when they are a waste of money","minutes":9}}}