The attacker is not the one typing

The threat model most teams are carrying is wrong

Ask a team how they handle prompt injection and you will usually hear about the input field: filters on the user’s message, checks for suspicious phrases, a classifier on what was typed.

That is direct injection, and it is the case that matters least. A user attacking your system through their own message is attacking their own session with their own permissions. They can usually do the thing directly.

The case that matters is indirect injection: the instruction is placed in content your system retrieves and puts in the window on the user’s behalf. The user typed something innocuous. The attack arrived through the door you built.

Every source is a channel

Write out the list of everything your assembler can pull into a window and mark who controls each one. The exercise takes fifteen minutes and it is uncomfortable.

• Fetched web pages. Controlled by whoever owns the site. Also by whoever can comment on it.
• Email. Controlled by anyone who knows the address. This is the starkest one: the attacker needs no access to your system at all, only an inbox to send to.
• Retrieved documents. Controlled by whoever can upload to the corpus: often, in a company deployment, any employee, or any customer with a support ticket.
• Support tickets, reviews, comments, form submissions, calendar invitations. All user-writable by design.
• Code repositories. Issue text, pull request descriptions, commit messages, dependency README files, and code comments in a package you installed.
• Search results. Controlled by whoever ranks for the query, which is a competitive and gameable position.
• Files on disk written earlier by a tool that fetched any of the above.
• Another model’s output, if a sub-agent read any of the above.

The last two are where careful teams get caught. The content laundered itself: it arrived from outside, passed through a tool or an agent, and now sits in a file or a summary that looks like your own working material.

What the payload looks like in practice

The folk image is a line reading “ignore all previous instructions”. Filters are written against that phrase, and it is the least of the problem, because the interesting payloads are not phrased as attacks at all.

What actually works is text that reads as legitimate system material: a note that appears to come from the document’s owner, a comment addressed to an automated processor, an instruction that fits the task the model is already doing, and text the human never sees.

That last one deserves its own list, because it is what makes the attack practical:

• White text on a white background in a PDF or a web page.
• HTML comments, alt attributes, metadata fields.
• Text rendered inside an image, when the model reads images.
• Content in a collapsed region, a footnote, or past a screenful of whitespace.
• Unicode that renders as nothing but tokenises as text.

The human reviewing the document sees a normal document. The extraction pipeline that produced the chunk sees the instruction and puts it in the window. Nobody in the loop has been given the chance to notice.

This is why “we have a human in the loop” is weaker than it sounds. A human approving an action is a real control. A human who glanced at the source document is not, because the channel was chosen precisely so they would not see it.

Nothing has to be compromised

The property that makes indirect injection hard to reason about is that it involves no breach anywhere.

The web page served exactly what its owner intended. The email was delivered normally. The document was uploaded through the supported route by someone with permission to upload. Your retrieval ranked it because it was relevant. Your model followed an instruction in its window, which is what models do.

Every component behaved correctly. There is no log line saying anything went wrong, no exception, no alert. If you go looking afterwards, the only trace is a tool call that looks a little unusual among thousands that look similar.

Which has a direct consequence for how you build: the detection has to be designed in, because it will not emerge from your existing monitoring. Logging which source each block in the window came from, and flagging actions taken in a turn where untrusted material was present, is not extra credit. It is the only way this shows up at all.

What to do on Monday

Before any defence, do the inventory. For every path by which text can reach a window in your system, write down who can control that text. Not who should, who can, including the adversarial case.

Most teams find at least one path they had never thought of as an input channel: the dependency README, the calendar invitation, the filename of an uploaded attachment. You cannot apply provenance rules to a source you have not noticed, and the next lessons are all about what you do once the list exists.