Many heads, and what has actually been found inside them

Why more than one

A single attention head produces one set of weights per position: one opinion about where to look. That is limiting. A position often needs several things at once: the subject of its clause, the opening bracket it must close, the last time this entity was mentioned.

Running many heads in parallel, each with its own learned projections, lets the layer gather several different kinds of information in one step. Each head operates on a slice of the width: a model with width 4,096 and 32 heads gives each head 128 dimensions to work in. Total compute is roughly the same as one full-width head, so the parallelism is nearly free.

Heads that have actually been identified

Interpretability research has found specific, verifiable circuits, mostly in small models studied deliberately. The best-established:

Previous-token heads. Attend almost entirely to the position immediately before. Simple, and a necessary component of the next one.

Induction heads. These implement a genuine algorithm: if the current token is A, find an earlier occurrence of A, and increase the probability of whatever followed it then. Formally, complete [A][B] ... [A] with [B]. This is how a model picks up on a made-up name or a pattern established earlier in your prompt, and it appears abruptly during training at the point where in-context learning ability jumps. It is the clearest link anyone has drawn between a mechanism and a capability.

Name-mover and inhibition heads. In the sentence “When Mary and John went to the shop, John gave a drink to ___”, researchers traced a circuit spanning several heads that identifies the candidate names, suppresses the one already used as the subject, and copies the other to the output. It was reverse-engineered in GPT-2 small and it holds up under intervention: knock out the inhibition head and the model produces the wrong name.

Copy-suppression heads. These reduce the probability of tokens already present nearby, which appears to be part of how models avoid repeating themselves.

The honest caveats

Now the corrections, which matter as much as the findings.

Most heads have no clean story. Publications show the interpretable ones, which is a selection effect. A given head in a large model usually does several partial things at once.

Heads are redundant. Removing a single head from a trained model often changes almost nothing, because several are doing overlapping work. This makes the “one head, one job” picture misleading even where the job is real.

Findings from small models may not transfer. Most circuit-level results come from models with millions to a few hundred million parameters. Large models are studied with coarser tools, and the assumption that the same circuits are there in the same form is a hypothesis, not a result.

The mistake to avoid: attention as explanation

You will see attention-weight heatmaps presented as explanations of model behaviour: “the model focused on the word not, which is why it classified the review as negative”. Treat this with suspicion. Two lines of research from 2019 showed that attention weights can often be substantially altered while leaving the output unchanged, and that different weightings can produce the same prediction. The debate that followed refined rather than overturned the point.

The reason is structural. Attention weights say where a value vector was gathered from. They say nothing about what the feed-forward layers then did with it, nothing about the residual stream contributions that bypassed attention entirely, and nothing about the other 31 heads in the same layer. A high weight is a plausible clue and not evidence.

What you can do with this

If you want to know whether a part of a model matters for a behaviour, the honest method is intervention, not observation: ablate it, or patch in activations from a different input, and see whether the output changes. That technique (activation patching) is the workhorse of the field and it is available free in open libraries such as TransformerLens, running on models small enough for a laptop.

The general principle transfers beyond interpretability. Watching where a system looks is weak evidence. Changing something and observing the difference is strong evidence.