Steering: pushing on a direction

The intervention the module has been building toward

If a feature is a direction, and if the model uses it, then adding a multiple of that direction to the residual stream during the forward pass should change the output in the way the feature’s meaning predicts. It does. This is activation steering, first demonstrated in 2023, and it is the cheapest experiment in this module with the largest implications.

Three ways to find a direction

Difference of means. Run a few hundred prompts that have the property and a few hundred that do not. Take the average activation at a chosen layer for each group and subtract. The result is a direction that separates them. It is crude, needs no training, and works far better than it has any right to.

A probe’s weights. The linear probe of the second lesson is a direction by construction.

A dictionary feature. The decoder column for a sparse-autoencoder feature from the third lesson.

All three give a vector in the residual stream’s space. Steering is the same whichever you used.

Doing it

# d: direction at layer L, from difference of means, normalised
d = (acts_pos[L].mean(0) - acts_neg[L].mean(0))
d = d / d.norm()

def steer(resid, hook):
    return resid + alpha * d          # add at every position

out = model.generate(prompt, fwd_hooks=[(f"blocks.{L}.hook_resid_post", steer)])

On a 2-billion-parameter open model in a free notebook this is about thirty lines including loading. Two knobs matter. The layer: middle layers usually steer best, because that is where abstract features live and there are enough layers after them to act on the change. The strength, alpha: too small and nothing happens; too large and the output dissolves into repetition. The usable range is a few multiples of the typical activation norm, found by trying five values.

Try it on formal versus casual register, on Hindi versus English output, on sycophantic versus blunt feedback. Each works, imperfectly, and each is a direction the model was already using.

The refusal direction, in full

Module six promised this. In 2024 researchers applied difference of means to thirteen open chat models: harmful instructions versus harmless ones, at a middle layer. Each model yielded a single direction. Subtracting it (projecting it out of the residual stream at every layer) produced a model that complied with almost any request while keeping its other capabilities. Adding it produced a model that refused to give a recipe for bread.

Folding the projection into the weights makes it permanent, with no training at all. Such “abliterated” versions of most popular open models now exist publicly, made in an afternoon each.

Read this carefully for what it does and does not say. It does not say safety training is useless; the direction only exists because training created it. It says that in these models refusal is one feature, thin and localisable, sitting on top of unchanged capability, and that every jailbreak in module six is an input that keeps that one direction from activating. Anyone deploying open weights should assume the feature can be removed, because it can, and put the actual safeguards where module four’s tool lesson put them.

Persona vectors and monitoring

In 2025 the same recipe was applied to character traits (sycophancy, hostility, a tendency to invent) producing directions that predict when a model is drifting toward the trait. Two uses followed. Scan candidate training data by measuring how far it pushes along a trait direction, and drop what pushes the wrong way. And steer against a trait during fine-tuning, so the model does not learn it from data that would otherwise teach it: a kind of vaccination. Vectors for emotional tone, truthfulness, verbosity and output language have all been reported by the same method.

The limits

Side effects. Directions are correlated. Steer toward honesty and the model gets terser; steer toward friendliness and it gets less accurate. You moved one direction and several features shared it, which is superposition again.

Brittleness. The layer and strength are tuned per model. The direction found for one version does not transfer to the next; module six’s drift applies to the model’s internals as much as to its outputs.

It is not training. Steering changes one forward pass and does nothing durable. For reliable behaviour change, training remains the tool; steering is for research, monitoring, quick experiments and, in a few reported cases, style control in production.

Closed models cannot be steered from outside. You need the activations. This is one of the concrete capability differences between hosting an open model and calling an API, and it is why the interpretability results in this module are almost all on open weights.

What it shows

Steering working at all is the strongest everyday evidence for the picture this module opened with. Features are directions; behaviour follows the direction; the model’s dispositions are things you can locate and push on. That is the sentence to keep, and the next lesson asks whether the same is true of the model’s facts.