Where the person goes

“A human approves everything” is not a safety design

It is a person doing the job slowly, with extra clicks. And it degrades on its own: show someone the twentieth near-identical approval dialog and they stop reading it. Approval fatigue is not a character flaw, it is what attention does. A gate that gets rubber-stamped is worse than no gate, because everyone now believes the work was checked.

So the design question is not whether to include a human. It is which decisions, and at what moment.

Gate on what cannot be undone

The useful axis is reversibility and blast radius. Not model confidence, more on that shortly.

This is a better guide than instinct, because instinct gates on how scary the action sounds. Deleting 12,000 rows from a table with point-in-time restore is fine. Sending one WhatsApp message to a customer is not undoable and deserves more care than it usually gets.

Three placements, three costs

Before the act: approval. The person sees the proposed action and says yes. Highest friction, highest assurance. Reserve it for the bottom-right box.

After the act: review queue plus undo. The agent acts, everything lands in a queue, someone skims and reverses what is wrong. Far cheaper, and it scales. It only works where undo is real. “We can email a correction” is not undo.

On the exception: the agent escalates itself. The best ratio when it works: the agent runs its own check, and hands over only what fails. The honest caveat is that this requires the agent to detect its own errors, which is precisely the thing it is worst at. Use it where the check is external and mechanical (arithmetic that must balance, an ID that must exist, a total that must match a receipt) and not where the check is the model marking its own homework.

Make the approval readable in three seconds

The shape of the request determines whether it is read.

Bad:

Agent wants to run refund_order. Approve? [y/N]

Good:

Refund ₹4,200 to arjun@example.com — order ORD-48213
Reason: item never shipped (courier shows no scan since 12 Aug)
This customer has had 0 previous refunds.

[Approve]  [Reject]  [Open the order]

The second one contains the facts a person needs to disagree. The first one only offers a decision, and a decision without evidence becomes a habit.

The trust ratchet

Start with everything gated. Log every single approval decision: what the agent proposed, what the human chose. After a few hundred, look at where the human agreed every time. Those are the categories you can safely ungate; you have evidence, not a feeling. Where the human disagreed even occasionally, the gate stays and you have just found your highest-value bug report.

Ratchet one direction only, and slowly. Removing a gate should require the same seriousness as deploying.

Do not gate on confidence

One trap worth naming directly. It is very tempting to have the model rate its own output and route the low scores to a human. It reads as elegant and it does not work, because self-reported confidence is not a measurement of correctness. A model is often most fluent, most assured and most detailed exactly when it is inventing: that is what confident error looks like from the inside. Route on category and consequence, which you can observe. Not on a number the model made up about itself.