What it means to know a fact: the reversal curse

The finding

In 2023 a group of researchers fine-tuned language models on a set of invented facts written in one direction: “Daphne Barrington is the director of A Journey Through Time.” Then they asked the reverse question: “Who directed A Journey Through Time?”

The models could not answer. Not badly: at chance. A model that had reliably learned “Daphne Barrington is the director of X” had learned nothing usable about “the director of X is Daphne Barrington”.

The same paper checked whether this held for facts learned in ordinary pretraining, using real celebrities. Asked “Who is Tom Cruise’s mother?”, GPT-4 answered correctly in the great majority of trials. Asked “Who is Mary Lee Pfeiffer’s son?”, it succeeded around a third of the time. The information is in there, in one direction.

They called it the reversal curse. It is a clean demonstration of something this course has been circling since module two: what a language model stores is not a fact. It is a directed association.

Why, in the mechanism you already have

Module two described the feed-forward half of each block as something like a key-value store. A pattern in the residual stream (the representation of “Daphne Barrington”) matches a key, and the corresponding value pushes the logits toward “director of A Journey Through Time”. That mapping was written by gradient descent, which module five showed nudges only the parameters that reduce loss on the sequences it actually saw.

The sequence it saw ran subject-to-object. Every update strengthened the path from the subject’s representation to the object’s tokens. No update ever strengthened a path from the object’s representation to the subject’s tokens, because no training sequence required one. The reverse key was never written, and there is no process in training that writes it as a side effect.

A database row is symmetric: store director(film) = Barrington and you can query it either way. A person who hears a fact once can usually reverse it, imperfectly. A language model’s parametric knowledge is neither. It is a bundle of one-way shortcuts, each running in the direction the text happened to be written.

Context knowledge is different

Put “Daphne Barrington directed A Journey Through Time” in the prompt and ask who directed it, and the model answers easily.

This is not a contradiction; it is the distinction the previous lesson drew. A fact in the context is reached through attention. When the question arrives, attention can look back to the sentence, find the film’s name, and copy the name beside it: the induction-style behaviour module two described. That works in either direction because the whole sentence is present and attention is a lookup over what is present.

So: in-context knowledge is bidirectional; weight knowledge is not. That single sentence explains a lot of otherwise puzzling behaviour and settles a lot of arguments about whether to fine-tune or retrieve.

What follows in practice

The direction the corpus was written in decides what is answerable. Text about a famous person is mostly written person-first, so “what did this person do” is easy and “who did this thing” is harder. Text about a paper names the paper and then its contribution, so “what did paper X introduce” works better than “which paper introduced X”, or the reverse, depending on how the field writes. Before assuming a model knows a domain, test both directions on twenty items. The asymmetry is often large.

Fine-tuning on your documents teaches them one way. Module five advised against fine-tuning to add facts. If you do it anyway, know that each fact goes in with the polarity of the sentence it came from. Teams that need both directions generate them: for each source sentence, write the reversed form and include it. Some labs now do this systematically during data preparation, precisely because of this result. It roughly doubles the data and it is the only known fix at the training stage.

Long-tail facts need many exposures, in each direction. Separate work has measured how the probability of a model knowing a fact rises with the number of times it appears in the corpus. The curve is steep at the bottom: something mentioned twice is mostly unknown, something mentioned a few hundred times is mostly known. Each direction counts separately.

Retrieval is the general answer. A fact in the window works both ways and was never subject to any of this. The previous lesson’s caveat still applies (memory can override context) but for facts the model has never seen, there is no memory to compete.

The honest state

The effect is robust across model sizes and families and has been replicated repeatedly. Scale does not appear to remove it; larger models know more facts in the forward direction and still fail the reverse at similar rates. Training objectives that read text in both directions, and data augmentation with reversals, reduce it. Nobody has shown it eliminated in a standard autoregressive model, and the mechanism above says why that would be surprising.

What the finding is not is evidence that models are useless at facts. It is evidence about the shape of what they hold: a large, directional, statistically weighted map of how text tends to continue, which behaves like a knowledge base often enough to be mistaken for one.