{"course":{"slug":"agents-and-automation","title":"Agents and Automation","url":"https://addaly.com/learn/agents-and-automation","level":"intermediate","track":"build"},"lesson":{"slug":"composing-patterns","title":"Joining two automations without joining their failures","url":"https://addaly.com/learn/agents-and-automation/composing-patterns","position":54,"of":72,"minutes":9,"answer":"The interface between two stages should be a queue, even if the queue is just a status column.","keyIdea":"The interface between two stages should be a queue, even if the queue is just a status column, so that a slow or broken stage produces a visible backlog instead of failures attributed to the wrong place."},"body":{"format":"markdown","text":"## Real systems are several patterns in a row\n\nAn invoice system is an extractor, then a validator, then a matcher against purchase orders, then a drafting step for the exceptions, then a monitor watching the whole thing. Five patterns from this module, joined.\n\nHow they are joined decides whether the result can be operated.\n\n## The interface is a queue, not a function call\n\nThe tempting design is direct: stage one finishes an invoice and calls stage two.\n\nNow suppose stage one extracts four thousand invoices an hour and stage two validates nine hundred. With a direct call, stage one blocks or times out, and its logs fill with errors. Somebody investigates the extractor, which is working perfectly. The real constraint is invisible, reported in the wrong place, and every failed call loses the extraction work that had already been paid for.\n\nWith a queue between them, stage one keeps running at its own speed and the backlog becomes a number on a chart that names the actual bottleneck. Stage two can be retried, paused, scaled or fixed without touching stage one. And a poisonous item can be set aside instead of stopping everything behind it.\n\n> Figure: Five stages, joined by a status column. Steps: 1. extracted (Stage one writes the row and stops. Four thousand an hour, at its own speed, blocked by nothing.); 2. validating (Claimed fifty at a time with FOR UPDATE SKIP LOCKED, by as many workers as you choose to run.); 3. validated or failed (A malformed PDF gets a reason and is set aside; the other four hundred carry on.); 4. matched (Against the purchase orders, keyed on the source reference so a replay changes nothing.); 5. drafted (Only the exceptions reach here, and a person sends.). Written as direct calls, a slow stage two shows up as errors in stage one's log and somebody spends a morning debugging a working extractor. Written as a status column, the backlog is a number that names the real bottleneck.\n\n**You do not need queueing infrastructure for this.** A status column is a queue:\n\n```sql\n-- stage 1 writes\nINSERT INTO invoices (id, raw_text, status) VALUES ($1, $2, 'extracted');\n\n-- stage 2 claims work\nUPDATE invoices SET status = 'validating', claimed_at = now()\nWHERE id IN (SELECT id FROM invoices WHERE status = 'extracted'\n             ORDER BY created_at LIMIT 50 FOR UPDATE SKIP LOCKED)\nRETURNING *;\n```\n\n`FOR UPDATE SKIP LOCKED` lets several workers claim different rows without colliding. That is a working job queue in a database you already run, with visibility, history and transactions, and it is the right answer far more often than a message broker.\n\n## Stages must not abort on one bad item\n\nOne malformed PDF should not stop the other four hundred. Mark the item `failed` with a reason, move on, and let failures accumulate somewhere a person reviews: a dead-letter queue, or just `status = 'failed'` with an error column.\n\nThen alert on the *rate* of failures, not on each one. Three failures an hour is normal; three hundred means the upstream format changed.\n\n## Own your columns\n\nEach stage writes only its own status values and its own fields. A stage that reaches into another's data creates a coupling nobody documented, and the day somebody changes stage two, stage four breaks for reasons that appear unrelated.\n\nWrite the state transitions down as a list (`extracted → validating → validated | failed`) and enforce them in code, refusing a transition that is not on the list. Ten lines, and it turns an entire class of confusing bug into an immediate exception naming the stage that misbehaved.\n\n## One id through everything\n\nGenerate a correlation id at the trigger and carry it into every stage, every log line, every outbound request and every row written. Tracing one invoice through five stages then becomes a single query instead of a morning.\n\n## Do not chain agents\n\nA pipeline where one stage is an agent is debuggable: the other stages are deterministic, so when the output is wrong you know where to look. A pipeline where every stage is an agent has no fixed point to reason from, per-step errors multiply across stages, and the trace is five transcripts that each look locally reasonable.\n\nPrefer deterministic stages with a model call inside the one or two that need judgement. It is cheaper, it is faster, and it is the difference between a system somebody can fix at 3am and a system somebody restarts and hopes.\n\n## Make every stage safe to run twice\n\nOnce stages are joined by a queue, items get delivered more than once: a worker that dies after doing the work but before marking it done, a claim that times out and is re-offered, a manual replay of yesterday's backlog. This is normal operation, not an error condition.\n\nSo each stage must be idempotent on its own: keyed writes, upserts on a natural key, and a check for \"have I already produced an output for this input id\" at the top of the handler. When every stage can be run twice harmlessly, recovery becomes trivial: you re-run the range and stop reasoning about exactly where it stopped. When one stage cannot, every incident turns into an archaeology exercise about which items got halfway.\n\n## Backpressure\n\nIf intake is faster than the slowest stage, the queue grows without limit until something breaks: usually disk, or a rate limit, or the bill. Decide the behaviour deliberately: cap intake, shed low-priority work, or scale the slow stage. Then alert on queue depth and on the age of the oldest unprocessed item, which is the number that tells you whether you are keeping up."},"check":{"question":"Stage one extracts 4,000 invoices an hour and calls stage two, which validates 900 an hour, directly. What is the main operational cost of that direct call?","options":[{"text":"Extraction results are lost on timeout, so the same invoices are processed repeatedly.","correct":false,"tempting":"Lost work is a real consequence, but it is a symptom of the coupling rather than the reason the problem is hard to operate."},{"text":"The bottleneck is hidden: stage one’s logs fill with errors it did not cause, so the investigation starts in the wrong place and the backlog is never visible as a number.","correct":true},{"text":"Stage two cannot be scaled horizontally while stage one holds an open connection to it.","correct":false,"tempting":"Connection handling complicates scaling, but you can add workers behind a load balancer; the deeper problem is that nothing shows the true constraint."},{"text":"Throughput is limited to the slower stage, so the system processes only 900 an hour overall.","correct":false,"tempting":"End-to-end throughput is bounded by the slowest stage with a queue too: the queue changes visibility and recoverability, not the ceiling."}]},"prev":{"slug":"the-coding-agent","title":"Pattern: the agent that writes and runs code","url":"https://addaly.com/learn/agents-and-automation/the-coding-agent"},"next":{"slug":"mcp-and-tool-standards","title":"MCP, and why a standard for tools matters","url":"https://addaly.com/learn/agents-and-automation/mcp-and-tool-standards"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Joining two automations without joining their failures', in Agents and Automation. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/agents-and-automation/composing-patterns (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Joining two automations without joining their failures","courseTitle":"Agents and Automation","module":{"index":6,"slug":"automation-patterns","title":"Eight patterns that keep working"},"answer":null,"keyIdea":"The interface between two stages should be a queue, even if the queue is just a status column, so that a slow or broken stage produces a visible backlog instead of failures attributed to the wrong place.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"flow","steps":[{"label":"extracted","detail":"Stage one writes the row and stops. Four thousand an hour, at its own speed, blocked by nothing."},{"label":"validating","detail":"Claimed fifty at a time with FOR UPDATE SKIP LOCKED, by as many workers as you choose to run."},{"label":"validated or failed","detail":"A malformed PDF gets a reason and is set aside; the other four hundred carry on."},{"label":"matched","detail":"Against the purchase orders, keyed on the source reference so a replay changes nothing."},{"label":"drafted","detail":"Only the exceptions reach here, and a person sends."}],"title":"Five stages, joined by a status column","caption":"Written as direct calls, a slow stage two shows up as errors in stage one’s log and somebody spends a morning debugging a working extractor. Written as a status column, the backlog is a number that names the real bottleneck."},"alt":"Figure: Five stages, joined by a status column. Steps: 1. extracted (Stage one writes the row and stops. Four thousand an hour, at its own speed, blocked by nothing.); 2. validating (Claimed fifty at a time with FOR UPDATE SKIP LOCKED, by as many workers as you choose to run.); 3. validated or failed (A malformed PDF gets a reason and is set aside; the other four hundred carry on.); 4. matched (Against the purchase orders, keyed on the source reference so a replay changes nothing.); 5. drafted (Only the exceptions reach here, and a person sends.). Written as direct calls, a slow stage two shows up as errors in stage one’s log and somebody spends a morning debugging a working extractor. Written as a status column, the backlog is a number that names the real bottleneck."}},"blocks":[{"t":"h2","id":"real-systems-are-several-patterns-in-a-row","runs":[{"text":"Real systems are several patterns in a row"}]},{"t":"p","runs":[{"text":"An invoice system is an extractor, then a validator, then a matcher against purchase orders, then a drafting step for the exceptions, then a monitor watching the whole thing. Five patterns from this module, joined."}]},{"t":"p","runs":[{"text":"How they are joined decides whether the result can be operated."}]},{"t":"h2","id":"the-interface-is-a-queue-not-a-function-call","runs":[{"text":"The interface is a queue, not a function call"}]},{"t":"p","runs":[{"text":"The tempting design is direct: stage one finishes an invoice and calls stage two."}]},{"t":"p","runs":[{"text":"Now suppose stage one extracts four thousand invoices an hour and stage two validates nine hundred. With a direct call, stage one blocks or times out, and its logs fill with errors. Somebody investigates the extractor, which is working perfectly. The real constraint is invisible, reported in the wrong place, and every failed call loses the extraction work that had already been paid for."}]},{"t":"p","runs":[{"text":"With a queue between them, stage one keeps running at its own speed and the backlog becomes a number on a chart that names the actual bottleneck. Stage two can be retried, paused, scaled or fixed without touching stage one. And a poisonous item can be set aside instead of stopping everything behind it."}]},{"t":"p","runs":[{"text":"You do not need queueing infrastructure for this.","b":true},{"text":" A status column is a queue:"}]},{"t":"code","lang":"sql","label":"SQL","text":"-- stage 1 writes\nINSERT INTO invoices (id, raw_text, status) VALUES ($1, $2, 'extracted');\n\n-- stage 2 claims work\nUPDATE invoices SET status = 'validating', claimed_at = now()\nWHERE id IN (SELECT id FROM invoices WHERE status = 'extracted'\n             ORDER BY created_at LIMIT 50 FOR UPDATE SKIP LOCKED)\nRETURNING *;"},{"t":"p","runs":[{"text":"FOR UPDATE SKIP LOCKED","code":true},{"text":" lets several workers claim different rows without colliding. That is a working job queue in a database you already run, with visibility, history and transactions, and it is the right answer far more often than a message broker."}]},{"t":"h2","id":"stages-must-not-abort-on-one-bad-item","runs":[{"text":"Stages must not abort on one bad item"}]},{"t":"p","runs":[{"text":"One malformed PDF should not stop the other four hundred. Mark the item "},{"text":"failed","code":true},{"text":" with a reason, move on, and let failures accumulate somewhere a person reviews: a dead-letter queue, or just "},{"text":"status = 'failed'","code":true},{"text":" with an error column."}]},{"t":"p","runs":[{"text":"Then alert on the "},{"text":"rate","i":true},{"text":" of failures, not on each one. Three failures an hour is normal; three hundred means the upstream format changed."}]},{"t":"h2","id":"own-your-columns","runs":[{"text":"Own your columns"}]},{"t":"p","runs":[{"text":"Each stage writes only its own status values and its own fields. A stage that reaches into another’s data creates a coupling nobody documented, and the day somebody changes stage two, stage four breaks for reasons that appear unrelated."}]},{"t":"p","runs":[{"text":"Write the state transitions down as a list ("},{"text":"extracted → validating → validated | failed","code":true},{"text":") and enforce them in code, refusing a transition that is not on the list. Ten lines, and it turns an entire class of confusing bug into an immediate exception naming the stage that misbehaved."}]},{"t":"h2","id":"one-id-through-everything","runs":[{"text":"One id through everything"}]},{"t":"p","runs":[{"text":"Generate a correlation id at the trigger and carry it into every stage, every log line, every outbound request and every row written. Tracing one invoice through five stages then becomes a single query instead of a morning."}]},{"t":"h2","id":"do-not-chain-agents","runs":[{"text":"Do not chain agents"}]},{"t":"p","runs":[{"text":"A pipeline where one stage is an agent is debuggable: the other stages are deterministic, so when the output is wrong you know where to look. A pipeline where every stage is an agent has no fixed point to reason from, per-step errors multiply across stages, and the trace is five transcripts that each look locally reasonable."}]},{"t":"p","runs":[{"text":"Prefer deterministic stages with a model call inside the one or two that need judgement. It is cheaper, it is faster, and it is the difference between a system somebody can fix at 3am and a system somebody restarts and hopes."}]},{"t":"h2","id":"make-every-stage-safe-to-run-twice","runs":[{"text":"Make every stage safe to run twice"}]},{"t":"p","runs":[{"text":"Once stages are joined by a queue, items get delivered more than once: a worker that dies after doing the work but before marking it done, a claim that times out and is re-offered, a manual replay of yesterday’s backlog. This is normal operation, not an error condition."}]},{"t":"p","runs":[{"text":"So each stage must be idempotent on its own: keyed writes, upserts on a natural key, and a check for “have I already produced an output for this input id” at the top of the handler. When every stage can be run twice harmlessly, recovery becomes trivial: you re-run the range and stop reasoning about exactly where it stopped. When one stage cannot, every incident turns into an archaeology exercise about which items got halfway."}]},{"t":"h2","id":"backpressure","runs":[{"text":"Backpressure"}]},{"t":"p","runs":[{"text":"If intake is faster than the slowest stage, the queue grows without limit until something breaks: usually disk, or a rate limit, or the bill. Decide the behaviour deliberately: cap intake, shed low-priority work, or scale the slow stage. Then alert on queue depth and on the age of the oldest unprocessed item, which is the number that tells you whether you are keeping up."}]}],"check":{"question":[{"text":"Stage one extracts 4,000 invoices an hour and calls stage two, which validates 900 an hour, directly. What is the main operational cost of that direct call?"}],"options":[{"text":[{"text":"Extraction results are lost on timeout, so the same invoices are processed repeatedly."}],"correct":false,"tempting":[{"text":"Lost work is a real consequence, but it is a symptom of the coupling rather than the reason the problem is hard to operate."}]},{"text":[{"text":"The bottleneck is hidden: stage one’s logs fill with errors it did not cause, so the investigation starts in the wrong place and the backlog is never visible as a number."}],"correct":true,"tempting":null},{"text":[{"text":"Stage two cannot be scaled horizontally while stage one holds an open connection to it."}],"correct":false,"tempting":[{"text":"Connection handling complicates scaling, but you can add workers behind a load balancer; the deeper problem is that nothing shows the true constraint."}]},{"text":[{"text":"Throughput is limited to the slower stage, so the system processes only 900 an hour overall."}],"correct":false,"tempting":[{"text":"End-to-end throughput is bounded by the slowest stage with a queue too: the queue changes visibility and recoverability, not the ceiling."}]}],"order":[3,0,2,1],"why":null},"prev":{"slug":"the-coding-agent","title":"Pattern: the agent that writes and runs code","minutes":10},"next":{"slug":"mcp-and-tool-standards","title":"MCP, and why a standard for tools matters","minutes":8}}}