{"course":{"slug":"how-llms-work","title":"How a Language Model Actually Works","url":"https://addaly.com/learn/how-llms-work","level":"intermediate","track":"start"},"lesson":{"slug":"the-reversal-curse","title":"What it means to know a fact: the reversal curse","url":"https://addaly.com/learn/how-llms-work/the-reversal-curse","position":53,"of":78,"minutes":8,"answer":"A fact in the weights is a one-way association written in the direction the training text ran, so a model can know that A is B without being able to answer what B is.","keyIdea":"A fact in the weights is a one-way association written in the direction the training text ran, so a model can know that A is B without being able to answer what B is, while a fact in the context works in both directions."},"body":{"format":"markdown","text":"## The finding\n\nIn 2023 a group of researchers fine-tuned language models on a set of invented facts written in one direction: \"Daphne Barrington is the director of *A Journey Through Time*.\" Then they asked the reverse question: \"Who directed *A Journey Through Time*?\"\n\nThe models could not answer. Not badly: at chance. A model that had reliably learned \"Daphne Barrington is the director of X\" had learned nothing usable about \"the director of X is Daphne Barrington\".\n\nThe same paper checked whether this held for facts learned in ordinary pretraining, using real celebrities. Asked \"Who is Tom Cruise's mother?\", GPT-4 answered correctly in the great majority of trials. Asked \"Who is Mary Lee Pfeiffer's son?\", it succeeded around a third of the time. The information is in there, in one direction.\n\nThey called it the reversal curse. It is a clean demonstration of something this course has been circling since module two: what a language model stores is not a fact. It is a directed association.\n\n## Why, in the mechanism you already have\n\nModule two described the feed-forward half of each block as something like a key-value store. A pattern in the residual stream (the representation of \"Daphne Barrington\") matches a key, and the corresponding value pushes the logits toward \"director of A Journey Through Time\". That mapping was written by gradient descent, which module five showed nudges only the parameters that reduce loss on the sequences it actually saw.\n\nThe sequence it saw ran subject-to-object. Every update strengthened the path from the subject's representation to the object's tokens. No update ever strengthened a path from the object's representation to the subject's tokens, because no training sequence required one. The reverse key was never written, and there is no process in training that writes it as a side effect.\n\nA database row is symmetric: store `director(film) = Barrington` and you can query it either way. A person who hears a fact once can usually reverse it, imperfectly. A language model's parametric knowledge is neither. It is a bundle of one-way shortcuts, each running in the direction the text happened to be written.\n\n## Context knowledge is different\n\nPut \"Daphne Barrington directed *A Journey Through Time*\" in the prompt and ask who directed it, and the model answers easily.\n\nThis is not a contradiction; it is the distinction the previous lesson drew. A fact in the context is reached through attention. When the question arrives, attention can look back to the sentence, find the film's name, and copy the name beside it: the induction-style behaviour module two described. That works in either direction because the whole sentence is present and attention is a lookup over what is present.\n\nSo: **in-context knowledge is bidirectional; weight knowledge is not.** That single sentence explains a lot of otherwise puzzling behaviour and settles a lot of arguments about whether to fine-tune or retrieve.\n\n## What follows in practice\n\n**The direction the corpus was written in decides what is answerable.** Text about a famous person is mostly written person-first, so \"what did this person do\" is easy and \"who did this thing\" is harder. Text about a paper names the paper and then its contribution, so \"what did paper X introduce\" works better than \"which paper introduced X\", or the reverse, depending on how the field writes. Before assuming a model knows a domain, test both directions on twenty items. The asymmetry is often large.\n\n**Fine-tuning on your documents teaches them one way.** Module five advised against fine-tuning to add facts. If you do it anyway, know that each fact goes in with the polarity of the sentence it came from. Teams that need both directions generate them: for each source sentence, write the reversed form and include it. Some labs now do this systematically during data preparation, precisely because of this result. It roughly doubles the data and it is the only known fix at the training stage.\n\n**Long-tail facts need many exposures, in each direction.** Separate work has measured how the probability of a model knowing a fact rises with the number of times it appears in the corpus. The curve is steep at the bottom: something mentioned twice is mostly unknown, something mentioned a few hundred times is mostly known. Each direction counts separately.\n\n**Retrieval is the general answer.** A fact in the window works both ways and was never subject to any of this. The previous lesson's caveat still applies (memory can override context) but for facts the model has never seen, there is no memory to compete.\n\n## The honest state\n\nThe effect is robust across model sizes and families and has been replicated repeatedly. Scale does not appear to remove it; larger models know more facts in the forward direction and still fail the reverse at similar rates. Training objectives that read text in both directions, and data augmentation with reversals, reduce it. Nobody has shown it eliminated in a standard autoregressive model, and the mechanism above says why that would be surprising.\n\nWhat the finding is not is evidence that models are useless at facts. It is evidence about the **shape** of what they hold: a large, directional, statistically weighted map of how text tends to continue, which behaves like a knowledge base often enough to be mistaken for one."},"check":{"question":"A team fine-tunes a model on 500 internal documents that each state \"Project Zeta's lead engineer is Priya Nair\". The model answers \"who leads Zeta?\" correctly and fails \"what does Priya Nair lead?\". Why?","options":[{"text":"Training strengthened a path from the subject's representation to the object's tokens, and nothing wrote the reverse path.","correct":true},{"text":"Five hundred documents is too few for a fact to generalise; a few thousand would cover the reverse direction as well.","correct":false,"tempting":"Reads the failure as under-training, but every one of the documents ran the same direction, so more of them would strengthen the same one-way path."},{"text":"The name is split into several tokens, and the model cannot reassemble a multi-token name into a subject it can query.","correct":false,"tempting":"Tokenisation does cause real failures, and the name is multi-token, but the model handles multi-token names as subjects all the time in the forward direction, so the split is not what is missing."},{"text":"Fine-tuning changes style rather than facts, so the fact was never actually learned and the correct forward answer is a coincidence with something in pretraining.","correct":false,"tempting":"Over-applies module five's guidance; fine-tuning does write facts, poorly and one-directionally, and the reliable forward answer on an internal project name is not a pretraining coincidence."}]},"prev":{"slug":"knowledge-in-weights-versus-context","title":"Two places a fact can live, and what happens when they disagree","url":"https://addaly.com/learn/how-llms-work/knowledge-in-weights-versus-context"},"next":{"slug":"reasoning-by-pattern","title":"Reasoning that is pattern matching, and how to tell","url":"https://addaly.com/learn/how-llms-work/reasoning-by-pattern"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What it means to know a fact: the reversal curse', in How a Language Model Actually Works. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/how-llms-work/the-reversal-curse (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What it means to know a fact: the reversal curse","courseTitle":"How a Language Model Actually Works","module":{"index":6,"slug":"why-it-goes-wrong","title":"Why it goes wrong, by mechanism"},"answer":null,"keyIdea":"A fact in the weights is a one-way association written in the direction the training text ran, so a model can know that A is B without being able to answer what B is, while a fact in the context works in both directions.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-finding","runs":[{"text":"The finding"}]},{"t":"p","runs":[{"text":"In 2023 a group of researchers fine-tuned language models on a set of invented facts written in one direction: “Daphne Barrington is the director of "},{"text":"A Journey Through Time","i":true},{"text":".” Then they asked the reverse question: “Who directed "},{"text":"A Journey Through Time","i":true},{"text":"?”"}]},{"t":"p","runs":[{"text":"The models could not answer. Not badly: at chance. A model that had reliably learned “Daphne Barrington is the director of X” had learned nothing usable about “the director of X is Daphne Barrington”."}]},{"t":"p","runs":[{"text":"The same paper checked whether this held for facts learned in ordinary pretraining, using real celebrities. Asked “Who is Tom Cruise’s mother?”, GPT-4 answered correctly in the great majority of trials. Asked “Who is Mary Lee Pfeiffer’s son?”, it succeeded around a third of the time. The information is in there, in one direction."}]},{"t":"p","runs":[{"text":"They called it the reversal curse. It is a clean demonstration of something this course has been circling since module two: what a language model stores is not a fact. It is a directed association."}]},{"t":"h2","id":"why-in-the-mechanism-you-already-have","runs":[{"text":"Why, in the mechanism you already have"}]},{"t":"p","runs":[{"text":"Module two described the feed-forward half of each block as something like a key-value store. A pattern in the residual stream (the representation of “Daphne Barrington”) matches a key, and the corresponding value pushes the logits toward “director of A Journey Through Time”. That mapping was written by gradient descent, which module five showed nudges only the parameters that reduce loss on the sequences it actually saw."}]},{"t":"p","runs":[{"text":"The sequence it saw ran subject-to-object. Every update strengthened the path from the subject’s representation to the object’s tokens. No update ever strengthened a path from the object’s representation to the subject’s tokens, because no training sequence required one. The reverse key was never written, and there is no process in training that writes it as a side effect."}]},{"t":"p","runs":[{"text":"A database row is symmetric: store "},{"text":"director(film) = Barrington","code":true},{"text":" and you can query it either way. A person who hears a fact once can usually reverse it, imperfectly. A language model’s parametric knowledge is neither. It is a bundle of one-way shortcuts, each running in the direction the text happened to be written."}]},{"t":"h2","id":"context-knowledge-is-different","runs":[{"text":"Context knowledge is different"}]},{"t":"p","runs":[{"text":"Put “Daphne Barrington directed "},{"text":"A Journey Through Time","i":true},{"text":"” in the prompt and ask who directed it, and the model answers easily."}]},{"t":"p","runs":[{"text":"This is not a contradiction; it is the distinction the previous lesson drew. A fact in the context is reached through attention. When the question arrives, attention can look back to the sentence, find the film’s name, and copy the name beside it: the induction-style behaviour module two described. That works in either direction because the whole sentence is present and attention is a lookup over what is present."}]},{"t":"p","runs":[{"text":"So: "},{"text":"in-context knowledge is bidirectional; weight knowledge is not.","b":true},{"text":" That single sentence explains a lot of otherwise puzzling behaviour and settles a lot of arguments about whether to fine-tune or retrieve."}]},{"t":"h2","id":"what-follows-in-practice","runs":[{"text":"What follows in practice"}]},{"t":"p","runs":[{"text":"The direction the corpus was written in decides what is answerable.","b":true},{"text":" Text about a famous person is mostly written person-first, so “what did this person do” is easy and “who did this thing” is harder. Text about a paper names the paper and then its contribution, so “what did paper X introduce” works better than “which paper introduced X”, or the reverse, depending on how the field writes. Before assuming a model knows a domain, test both directions on twenty items. The asymmetry is often large."}]},{"t":"p","runs":[{"text":"Fine-tuning on your documents teaches them one way.","b":true},{"text":" Module five advised against fine-tuning to add facts. If you do it anyway, know that each fact goes in with the polarity of the sentence it came from. Teams that need both directions generate them: for each source sentence, write the reversed form and include it. Some labs now do this systematically during data preparation, precisely because of this result. It roughly doubles the data and it is the only known fix at the training stage."}]},{"t":"p","runs":[{"text":"Long-tail facts need many exposures, in each direction.","b":true},{"text":" Separate work has measured how the probability of a model knowing a fact rises with the number of times it appears in the corpus. The curve is steep at the bottom: something mentioned twice is mostly unknown, something mentioned a few hundred times is mostly known. Each direction counts separately."}]},{"t":"p","runs":[{"text":"Retrieval is the general answer.","b":true},{"text":" A fact in the window works both ways and was never subject to any of this. The previous lesson’s caveat still applies (memory can override context) but for facts the model has never seen, there is no memory to compete."}]},{"t":"h2","id":"the-honest-state","runs":[{"text":"The honest state"}]},{"t":"p","runs":[{"text":"The effect is robust across model sizes and families and has been replicated repeatedly. Scale does not appear to remove it; larger models know more facts in the forward direction and still fail the reverse at similar rates. Training objectives that read text in both directions, and data augmentation with reversals, reduce it. Nobody has shown it eliminated in a standard autoregressive model, and the mechanism above says why that would be surprising."}]},{"t":"p","runs":[{"text":"What the finding is not is evidence that models are useless at facts. It is evidence about the "},{"text":"shape","b":true},{"text":" of what they hold: a large, directional, statistically weighted map of how text tends to continue, which behaves like a knowledge base often enough to be mistaken for one."}]}],"check":{"question":[{"text":"A team fine-tunes a model on 500 internal documents that each state “Project Zeta’s lead engineer is Priya Nair”. The model answers “who leads Zeta?” correctly and fails “what does Priya Nair lead?”. Why?"}],"options":[{"text":[{"text":"Training strengthened a path from the subject’s representation to the object’s tokens, and nothing wrote the reverse path."}],"correct":true,"tempting":null},{"text":[{"text":"Five hundred documents is too few for a fact to generalise; a few thousand would cover the reverse direction as well."}],"correct":false,"tempting":[{"text":"Reads the failure as under-training, but every one of the documents ran the same direction, so more of them would strengthen the same one-way path."}]},{"text":[{"text":"The name is split into several tokens, and the model cannot reassemble a multi-token name into a subject it can query."}],"correct":false,"tempting":[{"text":"Tokenisation does cause real failures, and the name is multi-token, but the model handles multi-token names as subjects all the time in the forward direction, so the split is not what is missing."}]},{"text":[{"text":"Fine-tuning changes style rather than facts, so the fact was never actually learned and the correct forward answer is a coincidence with something in pretraining."}],"correct":false,"tempting":[{"text":"Over-applies module five’s guidance; fine-tuning does write facts, poorly and one-directionally, and the reliable forward answer on an internal project name is not a pretraining coincidence."}]}],"order":[2,1,0,3],"why":null},"prev":{"slug":"knowledge-in-weights-versus-context","title":"Two places a fact can live, and what happens when they disagree","minutes":9},"next":{"slug":"reasoning-by-pattern","title":"Reasoning that is pattern matching, and how to tell","minutes":9}}}