Pattern: sort, enrich, escalate

The shape

A stream of items arrives: tickets, applications, alerts, emails, expense claims. The automation classifies each, attaches whatever context a person would have had to go and find, sets a priority, and puts it in a queue. People work the queue.

Nothing is decided and nothing is closed. That restraint is what makes the pattern durable.

Enrichment beats decision

The instinct is to make the system decide. The better investment is to make the decision easy.

Attach to each ticket: the customer’s last three orders, their previous two contacts, the relevant paragraph of the policy, the current status of the order in question, and whether they are within their return window. A person who would have spent four minutes gathering that now spends thirty seconds deciding.

The arithmetic is usually better than the decision-automation version, and the risk profile is completely different. Attaching the wrong policy paragraph is visible: the person reads it and sees it does not apply. Deciding wrongly is not visible, because the decision is the output.

Enrichment is also where a language model earns its place honestly: finding the relevant paragraph in a forty-page policy is exactly what it is good at, and the person retains the judgement about whether it applies.

The number that matters is the queue length

An automation that halves handling time and leaves the backlog growing has moved work, not removed it. Chart three things weekly: items in, items out, and the age of the oldest item. If the oldest item is getting older, everything else is decoration.

Sort by deadline, not by score. A model’s urgency rating is a guess; a service commitment made four days ago is a fact. Put ageing items at the top regardless of what the classifier thought, or you build a system where low-scored items are never touched at all: the queue’s quiet way of auto-closing things without anyone approving it.

A rules floor beneath the model

Some categories cannot be left to a score. Anything mentioning a regulator, legal action, a data breach, physical safety or self-harm goes straight to a person at top priority, matched by keyword, before the model runs.

Two reasons. The cost is asymmetric enough that a small false-positive rate is irrelevant. And a keyword list is auditable: you can show somebody the list, they can add to it, and it behaves the same way every time. When a regulator asks how you ensure such cases are seen promptly, “a rule that matches these forty terms” is an answer, and “the model usually prioritises them” is not.

The free labels

This is the pattern’s compounding advantage and most teams leave it on the floor.

Every time a person changes the category, moves an item to another queue or reprioritises it, they have produced a labelled example, for free, as a by-product of their job. Capture the before and after with the item text and you accumulate a training set worth thousands of pounds of annotation, at no cost.

Within a few months that set is enough to fine-tune a small model or train a cheap classifier for the high-volume categories, and to build the honest evaluation set for the whole system. Log it from day one, because you cannot reconstruct it retroactively: the field was overwritten.

Assign, do not only prioritise

A prioritised list still leaves everybody looking at the same pile and picking. Assignment turns triage into throughput: route to the queue that can act, respecting who is on shift, what language the customer wrote in, and who handled this customer last week. Continuity is worth more than skill matching in most support work, because the second contact about the same problem is where patience runs out.

Keep assignment reversible and visible. Anyone should be able to see why an item landed with them (the rule or label that routed it) and move it in one click without writing an explanation. Assignment that cannot be undone quietly becomes assignment that people stop trusting.

The failure everybody eventually proposes

At some point somebody suggests auto-closing the low-priority items, because nobody ever gets to them anyway.

The queue is full of things nobody got to, so this looks like tidying. What it does is convert a visible backlog into an invisible one, and the first time a genuine complaint is auto-closed, the trust cost lands on the whole system rather than on that decision. If the low-priority items are not worth answering, that is a policy the organisation should adopt openly, with a message to the customer saying so, not a threshold buried in a scoring function.