Limit what it can do, not what it can read

Move the control out of the prompt

Every defence so far has been probabilistic: warnings, delimiters, labels. They shift odds. None of them is a guarantee, and an attacker gets to keep trying.

The controls in this lesson are different in kind. They are conditions in your code, evaluated before the model is called, and the model has no vote. An attacker who fully controls the model’s output still cannot make a tool exist that you did not pass.

The premise is uncomfortable and clarifying: assume the injection succeeds. Design as though the attacker is writing the model’s output directly. What can they accomplish? The answer is exactly the set of tools available in that turn, with exactly the permissions those tools hold. Shrink that set and you have shrunk the attack, regardless of how persuasive the payload was.

Read-only by default, writes by exception

The first and largest reduction. Split every tool into read and write, where write means anything with an effect outside the process: sending, deleting, purchasing, posting, modifying a record, calling an external endpoint.

Then, using the provenance label from the previous lesson:

tools = all_tools
if ctx.has_untrusted:
    tools = [t for t in all_tools if t.read_only]

A turn that has read a web page or an inbound email can look things up and can answer. It cannot act. If the user’s request requires acting on something that was read, that becomes a second turn: one where the material has been reduced to a specific proposal the user confirmed, rather than a page of attacker-controlled prose.

This is a real product cost and it should be stated as one. It makes some flows two steps instead of one. It is also the single most effective control available, and it is about six lines.

Confirmation that shows the right thing

Where a write must be possible, put a human on it, but the design of the confirmation decides whether it is a control or a formality.

A dialogue saying “The assistant would like to send an email. Allow?” trains the user to click yes and protects nothing.

A useful confirmation shows the actual arguments, in full, and says what triggered them:

Send email
  To:      accounts@unfamiliar-domain.example
  Subject: Re: invoice
  Body:    [full text, not truncated]

This action was proposed while reading: inbox message from
stranger@example.net (untrusted).

The recipient is the field that catches exfiltration, because a strange destination is visible to a human in a way that a clever paragraph is not. Truncating the body to keep the dialogue tidy removes exactly the evidence somebody needs.

And confirmations must be rare. A user asked to approve forty actions approves the forty-first without reading. Batching many writes into one approval is worse than useless, because the odd one hides among the expected ones. If your design needs frequent confirmations, the answer is usually narrower tools, not more dialogues.

Constrain the arguments, not just the tool

A tool being available does not have to mean it is available for anything.

• fetch_url restricted to an allowlist of hosts.
• send_email restricted to addresses already in the user’s contacts, or to the sender of the message being replied to.
• run_sql restricted to a read-only connection against named views, with a row cap.
• write_file restricted to a working directory, with path traversal rejected rather than normalised.

Each of these turns an open channel into a closed one. The pattern is that the policy lives in the tool implementation, where it is testable and reviewable, rather than in a sentence hoping the model will comply.

Scope the data down too

The trifecta has three legs and capability limits attack one of them. While you are there, attack another.

An assistant with a search_all_documents tool has the whole corpus in scope on every turn. One scoped to the document the user selected has one document. The second cannot leak the first’s contents no matter what it is told, because it was never given them. Least privilege applied to retrieval scope is as effective here as it is anywhere else in security, and it is under-used because retrieval is usually built by people thinking about relevance rather than blast radius.

Bound the blast radius when everything fails

Assume all of the above is bypassed. What limits the damage?

• Rate limits on write tools, per user and per session. An attack that can send one email is bad. One that can send four hundred is a different event.
• Reversibility. Soft-delete rather than delete. Queue outbound messages with a short delay and a cancel. Make destructive actions require a second factor.
• Alerting on the signature. A write action in a turn that contained untrusted material is a rare and specific pattern, and you can only alert on it if you labelled the blocks. Alert on it.
• An audit trail that records which sources were in the window when an action was taken, so the question “which of our users were affected” has an answer.

The summary

None of this stops prompt injection. It is not meant to. It makes a successful injection into an event with bounded consequences, detectable afterwards, reversible in the common cases.

That is a lesser claim than a fix, and it is the claim that is currently true. A team that has done all of this is in a defensible position. A team that has written a strongly worded system prompt is not, however carefully it was worded.