Sparse autoencoders: a dictionary for the residual stream

The bet

Probes find the directions you can name. Superposition says there are thousands you cannot. The sparse autoencoder is a bet that they can be found anyway, without labels, from two assumptions the first lesson of this module set up: features are directions, and features are sparse.

The mechanism

Take activations from one layer of the model: say the 4,096-dimensional residual stream. Train a small network to reconstruct them through a hidden layer that is much wider than the input: 65,000 dimensions, or a million.

h      = relu( W_enc · x + b_enc )      # 4,096  -> 65,536, sparse
x_hat  = W_dec · h + b_dec              # 65,536 -> 4,096
loss   = |x - x_hat|²  +  λ · |h|₁

The first term wants a faithful reconstruction. The second penalises the hidden layer for having many non-zero entries. Together they say: explain each activation as a sum of as few dictionary entries as possible. Each column of the decoder is a candidate feature direction, and the sparsity is what pulls the overlapping directions of superposition apart into separate columns: without it, the autoencoder would learn the identity and tell you nothing. Some versions replace the penalty with “keep only the top k entries”, which does the same job more controllably.

Training data is tens of millions of activations from real text. For a small model this is a few hours on one GPU; for a production-scale model, days on many.

What was found

On a one-layer model in 2023, the dictionary contained thousands of features that were clean in a way neurons never were: Arabic script, DNA sequences, HTTP request headers, legal boilerplate, each firing on exactly what its name suggests.

Applied to production-scale models in 2024, the dictionaries reached millions of entries and the features became abstract. Code containing a bug. Sycophantic praise. A specific bridge. A specific person. Deception in a described scenario. Features that fire for the same concept across languages, and across text and images in a multimodal model: the Golden Gate Bridge feature fires on the opening sentence of the bridge’s Wikipedia article in several languages, and on photographs of the bridge, although the dictionary was trained only on text.

The demonstration that made the idea public was a model with one feature amplified (the bridge) that then brought the bridge into every answer, including its account of its own identity. Behaviour followed the direction, which is the steering lesson to come.

Two free resources make all of this something you can do rather than read about. Gemma Scope publishes trained dictionaries for every layer of an open 2-billion-parameter model. Neuronpedia lets you browse the features in a browser, see what each fires on, and run your own text through them. Loading a dictionary and listing the features active on a sentence is twenty lines with the sae_lens library in a free notebook.

Reading a feature

A feature is described three ways. Its top-activating examples (the passages where it fires hardest) give it a name. Its logit effect (which tokens become more likely when its direction is added) says what it does to the output. And steering (adding or removing it and watching the behaviour) says whether the name and the effect are real.

One property surprises everyone. Train a bigger dictionary on the same model and a single “birds” feature splits into species. Bigger again and the species split into contexts: a bird in a poem, a bird in a field guide. Features are not a fixed inventory the model has; they are a decomposition at a resolution you chose. The dictionary size is a dial, not a discovery.

The limits, which are the important part

Reconstruction is imperfect. Replace the model’s activations with the dictionary’s reconstruction and run the model: the loss rises measurably. Some fraction of the computation is not captured by any feature (the dictionary’s dark matter) and nobody can yet say what it does.

Many features resist interpretation, and some dictionary entries are dead, never firing, or dense, firing on everything. The count of clean features is impressive and it is a fraction of the dictionary.

Legibility is not mechanism. The dictionary is a basis chosen for our convenience. A feature that reads clearly to a person may not be a unit the model computes with; it may be a projection of several such units that happen to align with a human concept.

On downstream tasks the record is mixed. When dictionary features have been compared with plain linear probes on detection tasks (does this prompt contain a harmful request, is this output in the wrong language) probes have often matched or beaten them. Sparse autoencoders are a genuine advance in discovering what a model represents. They are not yet a reliable instrument for auditing it, and claims that a model has been “checked” with them should be read with that distinction in mind.

What you can do with one this week

Run your own prompts through a published dictionary and list the features that fire; it is the fastest way to see how a model carves up your domain. Use a feature as a cheap classifier in the forward pass. Check whether a “harmful request” feature fires on the jailbreak examples of module six, and whether the refusal that follows tracks it. None of that needs more than a free notebook, and it turns the previous four lessons into something you have seen with your own eyes.