{"course":{"slug":"agents-and-automation","title":"Agents and Automation","url":"https://addaly.com/learn/agents-and-automation","level":"intermediate","track":"build"},"lesson":{"slug":"the-triage-queue","title":"Pattern: sort, enrich, escalate","url":"https://addaly.com/learn/agents-and-automation/the-triage-queue","position":49,"of":72,"minutes":9,"answer":"Attaching the relevant evidence to an item is more valuable and far safer than deciding the item, and the human’s final disposition is a free training label you should be capturing.","keyIdea":"Attaching the relevant evidence to an item is more valuable and far safer than deciding the item, and the human’s final disposition is a free training label you should be capturing."},"body":{"format":"markdown","text":"## The shape\n\nA stream of items arrives: tickets, applications, alerts, emails, expense claims. The automation classifies each, attaches whatever context a person would have had to go and find, sets a priority, and puts it in a queue. People work the queue.\n\nNothing is decided and nothing is closed. That restraint is what makes the pattern durable.\n\n## Enrichment beats decision\n\nThe instinct is to make the system decide. The better investment is to make the decision easy.\n\nAttach to each ticket: the customer's last three orders, their previous two contacts, the relevant paragraph of the policy, the current status of the order in question, and whether they are within their return window. A person who would have spent four minutes gathering that now spends thirty seconds deciding.\n\nThe arithmetic is usually better than the decision-automation version, and the risk profile is completely different. Attaching the wrong policy paragraph is visible: the person reads it and sees it does not apply. Deciding wrongly is not visible, because the decision is the output.\n\nEnrichment is also where a language model earns its place honestly: finding the relevant paragraph in a forty-page policy is exactly what it is good at, and the person retains the judgement about whether it applies.\n\n## The number that matters is the queue length\n\nAn automation that halves handling time and leaves the backlog growing has moved work, not removed it. Chart three things weekly: items in, items out, and the age of the oldest item. If the oldest item is getting older, everything else is decoration.\n\nSort by deadline, not by score. A model's urgency rating is a guess; a service commitment made four days ago is a fact. Put ageing items at the top regardless of what the classifier thought, or you build a system where low-scored items are never touched at all: the queue's quiet way of auto-closing things without anyone approving it.\n\n## A rules floor beneath the model\n\nSome categories cannot be left to a score. Anything mentioning a regulator, legal action, a data breach, physical safety or self-harm goes straight to a person at top priority, matched by keyword, before the model runs.\n\nTwo reasons. The cost is asymmetric enough that a small false-positive rate is irrelevant. And a keyword list is auditable: you can show somebody the list, they can add to it, and it behaves the same way every time. When a regulator asks how you ensure such cases are seen promptly, \"a rule that matches these forty terms\" is an answer, and \"the model usually prioritises them\" is not.\n\n## The free labels\n\nThis is the pattern's compounding advantage and most teams leave it on the floor.\n\nEvery time a person changes the category, moves an item to another queue or reprioritises it, they have produced a labelled example, for free, as a by-product of their job. Capture the before and after with the item text and you accumulate a training set worth thousands of pounds of annotation, at no cost.\n\nWithin a few months that set is enough to fine-tune a small model or train a cheap classifier for the high-volume categories, and to build the honest evaluation set for the whole system. Log it from day one, because you cannot reconstruct it retroactively: the field was overwritten.\n\n## Assign, do not only prioritise\n\nA prioritised list still leaves everybody looking at the same pile and picking. Assignment turns triage into throughput: route to the queue that can act, respecting who is on shift, what language the customer wrote in, and who handled this customer last week. Continuity is worth more than skill matching in most support work, because the second contact about the same problem is where patience runs out.\n\nKeep assignment reversible and visible. Anyone should be able to see why an item landed with them (the rule or label that routed it) and move it in one click without writing an explanation. Assignment that cannot be undone quietly becomes assignment that people stop trusting.\n\n## The failure everybody eventually proposes\n\nAt some point somebody suggests auto-closing the low-priority items, because nobody ever gets to them anyway.\n\nThe queue is full of things nobody got to, so this looks like tidying. What it does is convert a visible backlog into an invisible one, and the first time a genuine complaint is auto-closed, the trust cost lands on the whole system rather than on that decision. If the low-priority items are not worth answering, that is a policy the organisation should adopt openly, with a message to the customer saying so, not a threshold buried in a scoring function."},"check":{"question":"A triage automation attaches the relevant policy paragraph and the customer’s order history to each ticket rather than recommending a resolution. Why is this both more valuable and safer than recommending?","options":[{"text":"It uses fewer tokens per ticket, so more tickets can be processed within the same budget.","correct":false,"tempting":"Retrieval and enrichment often cost more than a short recommendation, so the case does not rest on cost."},{"text":"It avoids making a decision the system is not authorised to make, so the compliance position is simpler.","correct":false,"tempting":"Authority is a real consideration, but a recommendation a human approves is also authorised; this misses why the enrichment is more useful."},{"text":"It keeps the human in the loop, which is required whenever a model influences an outcome.","correct":false,"tempting":"States a general principle rather than the mechanism, and enrichment influences outcomes too; the difference is that its errors surface on sight."},{"text":"It removes the gathering work, which is where the minutes are, and a wrong attachment is visible to the reader in a way a wrong decision is not.","correct":true}]},"prev":{"slug":"the-drafting-assistant","title":"Pattern: it writes, a person sends","url":"https://addaly.com/learn/agents-and-automation/the-drafting-assistant"},"next":{"slug":"the-research-agent","title":"Pattern: search, read, cite, and verify the citation","url":"https://addaly.com/learn/agents-and-automation/the-research-agent"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Pattern: sort, enrich, escalate', in Agents and Automation. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/agents-and-automation/the-triage-queue (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Pattern: sort, enrich, escalate","courseTitle":"Agents and Automation","module":{"index":6,"slug":"automation-patterns","title":"Eight patterns that keep working"},"answer":null,"keyIdea":"Attaching the relevant evidence to an item is more valuable and far safer than deciding the item, and the human’s final disposition is a free training label you should be capturing.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-shape","runs":[{"text":"The shape"}]},{"t":"p","runs":[{"text":"A stream of items arrives: tickets, applications, alerts, emails, expense claims. The automation classifies each, attaches whatever context a person would have had to go and find, sets a priority, and puts it in a queue. People work the queue."}]},{"t":"p","runs":[{"text":"Nothing is decided and nothing is closed. That restraint is what makes the pattern durable."}]},{"t":"h2","id":"enrichment-beats-decision","runs":[{"text":"Enrichment beats decision"}]},{"t":"p","runs":[{"text":"The instinct is to make the system decide. The better investment is to make the decision easy."}]},{"t":"p","runs":[{"text":"Attach to each ticket: the customer’s last three orders, their previous two contacts, the relevant paragraph of the policy, the current status of the order in question, and whether they are within their return window. A person who would have spent four minutes gathering that now spends thirty seconds deciding."}]},{"t":"p","runs":[{"text":"The arithmetic is usually better than the decision-automation version, and the risk profile is completely different. Attaching the wrong policy paragraph is visible: the person reads it and sees it does not apply. Deciding wrongly is not visible, because the decision is the output."}]},{"t":"p","runs":[{"text":"Enrichment is also where a language model earns its place honestly: finding the relevant paragraph in a forty-page policy is exactly what it is good at, and the person retains the judgement about whether it applies."}]},{"t":"h2","id":"the-number-that-matters-is-the-queue-length","runs":[{"text":"The number that matters is the queue length"}]},{"t":"p","runs":[{"text":"An automation that halves handling time and leaves the backlog growing has moved work, not removed it. Chart three things weekly: items in, items out, and the age of the oldest item. If the oldest item is getting older, everything else is decoration."}]},{"t":"p","runs":[{"text":"Sort by deadline, not by score. A model’s urgency rating is a guess; a service commitment made four days ago is a fact. Put ageing items at the top regardless of what the classifier thought, or you build a system where low-scored items are never touched at all: the queue’s quiet way of auto-closing things without anyone approving it."}]},{"t":"h2","id":"a-rules-floor-beneath-the-model","runs":[{"text":"A rules floor beneath the model"}]},{"t":"p","runs":[{"text":"Some categories cannot be left to a score. Anything mentioning a regulator, legal action, a data breach, physical safety or self-harm goes straight to a person at top priority, matched by keyword, before the model runs."}]},{"t":"p","runs":[{"text":"Two reasons. The cost is asymmetric enough that a small false-positive rate is irrelevant. And a keyword list is auditable: you can show somebody the list, they can add to it, and it behaves the same way every time. When a regulator asks how you ensure such cases are seen promptly, “a rule that matches these forty terms” is an answer, and “the model usually prioritises them” is not."}]},{"t":"h2","id":"the-free-labels","runs":[{"text":"The free labels"}]},{"t":"p","runs":[{"text":"This is the pattern’s compounding advantage and most teams leave it on the floor."}]},{"t":"p","runs":[{"text":"Every time a person changes the category, moves an item to another queue or reprioritises it, they have produced a labelled example, for free, as a by-product of their job. Capture the before and after with the item text and you accumulate a training set worth thousands of pounds of annotation, at no cost."}]},{"t":"p","runs":[{"text":"Within a few months that set is enough to fine-tune a small model or train a cheap classifier for the high-volume categories, and to build the honest evaluation set for the whole system. Log it from day one, because you cannot reconstruct it retroactively: the field was overwritten."}]},{"t":"h2","id":"assign-do-not-only-prioritise","runs":[{"text":"Assign, do not only prioritise"}]},{"t":"p","runs":[{"text":"A prioritised list still leaves everybody looking at the same pile and picking. Assignment turns triage into throughput: route to the queue that can act, respecting who is on shift, what language the customer wrote in, and who handled this customer last week. Continuity is worth more than skill matching in most support work, because the second contact about the same problem is where patience runs out."}]},{"t":"p","runs":[{"text":"Keep assignment reversible and visible. Anyone should be able to see why an item landed with them (the rule or label that routed it) and move it in one click without writing an explanation. Assignment that cannot be undone quietly becomes assignment that people stop trusting."}]},{"t":"h2","id":"the-failure-everybody-eventually-proposes","runs":[{"text":"The failure everybody eventually proposes"}]},{"t":"p","runs":[{"text":"At some point somebody suggests auto-closing the low-priority items, because nobody ever gets to them anyway."}]},{"t":"p","runs":[{"text":"The queue is full of things nobody got to, so this looks like tidying. What it does is convert a visible backlog into an invisible one, and the first time a genuine complaint is auto-closed, the trust cost lands on the whole system rather than on that decision. If the low-priority items are not worth answering, that is a policy the organisation should adopt openly, with a message to the customer saying so, not a threshold buried in a scoring function."}]}],"check":{"question":[{"text":"A triage automation attaches the relevant policy paragraph and the customer’s order history to each ticket rather than recommending a resolution. Why is this both more valuable and safer than recommending?"}],"options":[{"text":[{"text":"It uses fewer tokens per ticket, so more tickets can be processed within the same budget."}],"correct":false,"tempting":[{"text":"Retrieval and enrichment often cost more than a short recommendation, so the case does not rest on cost."}]},{"text":[{"text":"It avoids making a decision the system is not authorised to make, so the compliance position is simpler."}],"correct":false,"tempting":[{"text":"Authority is a real consideration, but a recommendation a human approves is also authorised; this misses why the enrichment is more useful."}]},{"text":[{"text":"It keeps the human in the loop, which is required whenever a model influences an outcome."}],"correct":false,"tempting":[{"text":"States a general principle rather than the mechanism, and enrichment influences outcomes too; the difference is that its errors surface on sight."}]},{"text":[{"text":"It removes the gathering work, which is where the minutes are, and a wrong attachment is visible to the reader in a way a wrong decision is not."}],"correct":true,"tempting":null}],"order":[1,2,0,3],"why":null},"prev":{"slug":"the-drafting-assistant","title":"Pattern: it writes, a person sends","minutes":9},"next":{"slug":"the-research-agent","title":"Pattern: search, read, cite, and verify the citation","minutes":10}}}