{"course":{"slug":"how-llms-work","title":"How a Language Model Actually Works","url":"https://addaly.com/learn/how-llms-work","level":"intermediate","track":"start"},"lesson":{"slug":"superposition-and-features","title":"Superposition: more features than dimensions","url":"https://addaly.com/learn/how-llms-work/superposition-and-features","position":69,"of":78,"minutes":9,"answer":"A model stores more features than it has dimensions by placing them along nearly orthogonal directions that rarely fire together.","keyIdea":"A model stores more features than it has dimensions by placing them along nearly orthogonal directions that rarely fire together, which is why single neurons look meaningless and why every method for reading a model has to find directions instead."},"body":{"format":"markdown","text":"## The puzzle left over from module two\n\nThe feed-forward lesson reported that individual neurons respond to unrelated collections of things (legal citations, chess notation, a kind of German compound) and named the leading explanation, superposition, in a paragraph. This module needs more than a paragraph, because every method in it depends on what superposition implies about **where to look**.\n\n## What a feature is\n\nA feature is a property of the input that the model tracks: this text is code, this token is a person's name, this sentence is in French, the statement just made was negated, the previous word was an adjective. Operationally, a feature is a **direction** in the space of activations. Project the residual-stream vector onto that direction and you get a number that reads as how strongly the feature is present.\n\nThat definition carries an assumption, usually called the linear representation hypothesis: that features are directions and their strength is a projection, so that adding two features' directions represents both at once. It is an assumption with a great deal of evidence behind it and some known exceptions, and this module treats it as a working model rather than a law.\n\n## Why a model would pack in more features than it has room for\n\nThe residual stream of an 8-billion-parameter model has 4,096 dimensions. The model tracks far more than 4,096 distinguishable things about text. So features cannot each have a dimension to themselves.\n\nThe way out is geometry. In 4,096 dimensions you can place enormously many vectors that are *nearly* orthogonal: pairwise dot products small but not zero. If features are **sparse**, meaning that any given input activates only a few of them, then storing them along nearly orthogonal directions works: the interference between features that are rarely active together is small noise, and the model gains capacity in exchange. That arrangement is superposition, and it means a neuron, one basis direction, is crossed by many features, and each feature is spread across many neurons.\n\n## The toy model\n\nA 2022 paper made this reproducible. Train a tiny network to reconstruct *n* sparse features through a bottleneck of *m* dimensions, with *m* smaller than *n*, and vary how sparse the features are.\n\n```python\n# sketch: n features, m < n dimensions, sparsity s\nx = (torch.rand(batch, n) < s) * torch.rand(batch, n)   # sparse inputs\nh = x @ W                                                # n -> m\nx_hat = torch.relu(h @ W.T + b)                          # m -> n\nloss = ((x - x_hat) ** 2 * importance).sum()\n```\n\nWhen features are dense, active most of the time, the network keeps exactly *m* of them, one per dimension, and drops the rest. As sparsity increases it starts packing more than *m* features in, arranging their directions into regular geometric shapes (pairs, triangles, pentagons) that minimise interference. Nothing told it to do this; it is the optimal solution to \"represent as much as possible with limited room and rare co-occurrence\". The whole experiment runs in a free notebook in minutes, and it is worth running once, because it turns a slogan into something you have watched happen.\n\n> Figure: The toy model: what the network chooses to store. Line chart of Features represented, per bottleneck dimension against Fraction of features active on a given input, %: n features through m dimensions, m smaller than n (0.3, 9.2) (1, 5) (3, 2.6) (10, 1.4) (30, 1) (100, 1). With dense features the network keeps exactly one per dimension and drops the rest. As features become rarer it packs in more than it has room for, arranging the directions into shapes (pairs, triangles, pentagons) that keep the interference small. Nothing told it to. The experiment runs in a free notebook in minutes.\n\n## Four consequences\n\n1. **Neurons are the wrong unit.** A neuron is one axis of the coordinate system. Features live along other directions. Asking what a neuron does is asking what several unrelated features have in common, which is why the answer is \"legal citations and chess\".\n2. **Interference is a source of error.** When two features that share directions are active together, each reads out slightly wrong. This is a candidate mechanism for some of the strange, input-specific mistakes of module six: the model's own bookkeeping is lossy by design.\n3. **Interpretation means finding directions.** The next three lessons are three ways of doing that: ask the activations a labelled question, learn a dictionary of directions without labels, and intervene to see which directions matter.\n4. **Width helps, partly.** A larger residual stream can hold more features with less crowding, which is one reason larger models are cleaner to interpret per feature. It does not remove the effect, because the number of features the model wants to track grows too.\n\n## The evidence, and the edges of it\n\nThe support is now considerable. Linear probes find features as directions, and the same directions turn up in different models. Sparse dictionaries recover thousands of clean, single-meaning features from activations that looked hopeless at the neuron level. Adding a direction to the activations changes behaviour in the way the feature's meaning predicts. Each of those is a lesson to come.\n\nThe exceptions are real. In 2024 features were found that are not single directions but small circular structures: the days of the week arranged in a ring, so that \"Tuesday plus two\" lands on Thursday. Some features appear to be represented in low-dimensional subspaces rather than lines. The hypothesis that everything is a direction is therefore incomplete, and the honest position is that most of what has been found is linear, some of it is not, and nobody knows the proportions.\n\n## Why this matters outside research\n\nBecause the rest of this module is only usable if you have this picture. When a later lesson says \"the refusal direction\" or \"a truth probe\" or \"amplify the feature\", it means a direction in activation space that a model packed in among thousands of others. That is what you are reading, what you are steering, and what you would have to remove to unlearn something. Every claim about what a model \"knows\" that you will meet from here on is a claim about such directions, and how well they can be found."},"check":{"question":"A neuron in a large model fires on legal citations, chess notation and a certain kind of German compound noun. Why does one neuron respond to three unrelated things?","options":[{"text":"The training data happened to contain those three things together in the same documents, so the model associated them.","correct":false,"tempting":"Co-occurrence in data does shape representations, but these features fire on entirely separate documents; the neuron is not tracking an association between them."},{"text":"The model stores more features than it has dimensions, so each neuron's direction is shared among features that rarely co-occur.","correct":true},{"text":"Some neurons never specialise during training and remain close to their random initialisation, responding to whatever happens to project onto them.","correct":false,"tempting":"Unspecialised neurons exist, but this neuron responds sharply and reliably to each of its three triggers, which is the signature of features packed in, not of a neuron left random."},{"text":"The three share a hidden property, compact formal notation, that the neuron has correctly learned as one concept, and researchers have simply not yet identified the unifying description that makes the neuron single-purpose.","correct":false,"tempting":"Occasionally true of an individual neuron, and always worth checking, but the pattern is systematic across most neurons in every model examined, which is what superposition predicts and a hidden common concept does not."}]},"prev":{"slug":"beyond-the-transformer","title":"Beyond the transformer: what the alternatives trade","url":"https://addaly.com/learn/how-llms-work/beyond-the-transformer"},"next":{"slug":"probing-what-is-represented","title":"Probing: asking the activations a question","url":"https://addaly.com/learn/how-llms-work/probing-what-is-represented"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Superposition: more features than dimensions', in How a Language Model Actually Works. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/how-llms-work/superposition-and-features (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Superposition: more features than dimensions","courseTitle":"How a Language Model Actually Works","module":{"index":8,"slug":"looking-inside","title":"Looking inside: what the weights actually hold"},"answer":null,"keyIdea":"A model stores more features than it has dimensions by placing them along nearly orthogonal directions that rarely fire together, which is why single neurons look meaningless and why every method for reading a model has to find directions instead.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure 8.1","figure":{"type":"line","title":"The toy model: what the network chooses to store","series":[{"label":"n features through m dimensions, m smaller than n","points":[[0.3,9.2],[1,5],[3,2.6],[10,1.4],[30,1],[100,1]]}],"xLabel":"Fraction of features active on a given input, %","yLabel":"Features represented, per bottleneck dimension","caption":"With dense features the network keeps exactly one per dimension and drops the rest. As features become rarer it packs in more than it has room for, arranging the directions into shapes (pairs, triangles, pentagons) that keep the interference small. Nothing told it to. The experiment runs in a free notebook in minutes."},"alt":"Figure: The toy model: what the network chooses to store. Line chart of Features represented, per bottleneck dimension against Fraction of features active on a given input, %: n features through m dimensions, m smaller than n (0.3, 9.2) (1, 5) (3, 2.6) (10, 1.4) (30, 1) (100, 1). With dense features the network keeps exactly one per dimension and drops the rest. As features become rarer it packs in more than it has room for, arranging the directions into shapes (pairs, triangles, pentagons) that keep the interference small. Nothing told it to. The experiment runs in a free notebook in minutes."}},"blocks":[{"t":"h2","id":"the-puzzle-left-over-from-module-two","runs":[{"text":"The puzzle left over from module two"}]},{"t":"p","runs":[{"text":"The feed-forward lesson reported that individual neurons respond to unrelated collections of things (legal citations, chess notation, a kind of German compound) and named the leading explanation, superposition, in a paragraph. This module needs more than a paragraph, because every method in it depends on what superposition implies about "},{"text":"where to look","b":true},{"text":"."}]},{"t":"h2","id":"what-a-feature-is","runs":[{"text":"What a feature is"}]},{"t":"p","runs":[{"text":"A feature is a property of the input that the model tracks: this text is code, this token is a person’s name, this sentence is in French, the statement just made was negated, the previous word was an adjective. Operationally, a feature is a "},{"text":"direction","b":true},{"text":" in the space of activations. Project the residual-stream vector onto that direction and you get a number that reads as how strongly the feature is present."}]},{"t":"p","runs":[{"text":"That definition carries an assumption, usually called the linear representation hypothesis: that features are directions and their strength is a projection, so that adding two features’ directions represents both at once. It is an assumption with a great deal of evidence behind it and some known exceptions, and this module treats it as a working model rather than a law."}]},{"t":"h2","id":"why-a-model-would-pack-in-more-features-than-it-has-room-for","runs":[{"text":"Why a model would pack in more features than it has room for"}]},{"t":"p","runs":[{"text":"The residual stream of an 8-billion-parameter model has 4,096 dimensions. The model tracks far more than 4,096 distinguishable things about text. So features cannot each have a dimension to themselves."}]},{"t":"p","runs":[{"text":"The way out is geometry. In 4,096 dimensions you can place enormously many vectors that are "},{"text":"nearly","i":true},{"text":" orthogonal: pairwise dot products small but not zero. If features are "},{"text":"sparse","b":true},{"text":", meaning that any given input activates only a few of them, then storing them along nearly orthogonal directions works: the interference between features that are rarely active together is small noise, and the model gains capacity in exchange. That arrangement is superposition, and it means a neuron, one basis direction, is crossed by many features, and each feature is spread across many neurons."}]},{"t":"h2","id":"the-toy-model","runs":[{"text":"The toy model"}]},{"t":"p","runs":[{"text":"A 2022 paper made this reproducible. Train a tiny network to reconstruct "},{"text":"n","i":true},{"text":" sparse features through a bottleneck of "},{"text":"m","i":true},{"text":" dimensions, with "},{"text":"m","i":true},{"text":" smaller than "},{"text":"n","i":true},{"text":", and vary how sparse the features are."}]},{"t":"code","lang":"python","label":"Python","text":"# sketch: n features, m < n dimensions, sparsity s\nx = (torch.rand(batch, n) < s) * torch.rand(batch, n)   # sparse inputs\nh = x @ W                                                # n -> m\nx_hat = torch.relu(h @ W.T + b)                          # m -> n\nloss = ((x - x_hat) ** 2 * importance).sum()"},{"t":"p","runs":[{"text":"When features are dense, active most of the time, the network keeps exactly "},{"text":"m","i":true},{"text":" of them, one per dimension, and drops the rest. As sparsity increases it starts packing more than "},{"text":"m","i":true},{"text":" features in, arranging their directions into regular geometric shapes (pairs, triangles, pentagons) that minimise interference. Nothing told it to do this; it is the optimal solution to “represent as much as possible with limited room and rare co-occurrence”. The whole experiment runs in a free notebook in minutes, and it is worth running once, because it turns a slogan into something you have watched happen."}]},{"t":"h2","id":"four-consequences","runs":[{"text":"Four consequences"}]},{"t":"ol","start":1,"items":[[{"text":"Neurons are the wrong unit.","b":true},{"text":" A neuron is one axis of the coordinate system. Features live along other directions. Asking what a neuron does is asking what several unrelated features have in common, which is why the answer is “legal citations and chess”."}],[{"text":"Interference is a source of error.","b":true},{"text":" When two features that share directions are active together, each reads out slightly wrong. This is a candidate mechanism for some of the strange, input-specific mistakes of module six: the model’s own bookkeeping is lossy by design."}],[{"text":"Interpretation means finding directions.","b":true},{"text":" The next three lessons are three ways of doing that: ask the activations a labelled question, learn a dictionary of directions without labels, and intervene to see which directions matter."}],[{"text":"Width helps, partly.","b":true},{"text":" A larger residual stream can hold more features with less crowding, which is one reason larger models are cleaner to interpret per feature. It does not remove the effect, because the number of features the model wants to track grows too."}]]},{"t":"h2","id":"the-evidence-and-the-edges-of-it","runs":[{"text":"The evidence, and the edges of it"}]},{"t":"p","runs":[{"text":"The support is now considerable. Linear probes find features as directions, and the same directions turn up in different models. Sparse dictionaries recover thousands of clean, single-meaning features from activations that looked hopeless at the neuron level. Adding a direction to the activations changes behaviour in the way the feature’s meaning predicts. Each of those is a lesson to come."}]},{"t":"p","runs":[{"text":"The exceptions are real. In 2024 features were found that are not single directions but small circular structures: the days of the week arranged in a ring, so that “Tuesday plus two” lands on Thursday. Some features appear to be represented in low-dimensional subspaces rather than lines. The hypothesis that everything is a direction is therefore incomplete, and the honest position is that most of what has been found is linear, some of it is not, and nobody knows the proportions."}]},{"t":"h2","id":"why-this-matters-outside-research","runs":[{"text":"Why this matters outside research"}]},{"t":"p","runs":[{"text":"Because the rest of this module is only usable if you have this picture. When a later lesson says “the refusal direction” or “a truth probe” or “amplify the feature”, it means a direction in activation space that a model packed in among thousands of others. That is what you are reading, what you are steering, and what you would have to remove to unlearn something. Every claim about what a model “knows” that you will meet from here on is a claim about such directions, and how well they can be found."}]}],"check":{"question":[{"text":"A neuron in a large model fires on legal citations, chess notation and a certain kind of German compound noun. Why does one neuron respond to three unrelated things?"}],"options":[{"text":[{"text":"The training data happened to contain those three things together in the same documents, so the model associated them."}],"correct":false,"tempting":[{"text":"Co-occurrence in data does shape representations, but these features fire on entirely separate documents; the neuron is not tracking an association between them."}]},{"text":[{"text":"The model stores more features than it has dimensions, so each neuron’s direction is shared among features that rarely co-occur."}],"correct":true,"tempting":null},{"text":[{"text":"Some neurons never specialise during training and remain close to their random initialisation, responding to whatever happens to project onto them."}],"correct":false,"tempting":[{"text":"Unspecialised neurons exist, but this neuron responds sharply and reliably to each of its three triggers, which is the signature of features packed in, not of a neuron left random."}]},{"text":[{"text":"The three share a hidden property, compact formal notation, that the neuron has correctly learned as one concept, and researchers have simply not yet identified the unifying description that makes the neuron single-purpose."}],"correct":false,"tempting":[{"text":"Occasionally true of an individual neuron, and always worth checking, but the pattern is systematic across most neurons in every model examined, which is what superposition predicts and a hidden common concept does not."}]}],"order":[0,3,1,2],"why":null},"prev":{"slug":"beyond-the-transformer","title":"Beyond the transformer: what the alternatives trade","minutes":9},"next":{"slug":"probing-what-is-represented","title":"Probing: asking the activations a question","minutes":9}}}