{"course":{"slug":"how-llms-work","title":"How a Language Model Actually Works","url":"https://addaly.com/learn/how-llms-work","level":"intermediate","track":"start"},"lesson":{"slug":"steering-with-directions","title":"Steering: pushing on a direction","url":"https://addaly.com/learn/how-llms-work/steering-with-directions","position":73,"of":78,"minutes":9,"answer":"Adding a feature's direction to the residual stream changes behaviour in the way the feature's meaning predicts.","keyIdea":"Adding a feature's direction to the residual stream changes behaviour in the way the feature's meaning predicts, and the discovery that refusal in open chat models is one such direction (removable in an afternoon) is why the model's own refusals cannot be the safety boundary."},"body":{"format":"markdown","text":"## The intervention the module has been building toward\n\nIf a feature is a direction, and if the model uses it, then adding a multiple of that direction to the residual stream during the forward pass should change the output in the way the feature's meaning predicts. It does. This is activation steering, first demonstrated in 2023, and it is the cheapest experiment in this module with the largest implications.\n\n## Three ways to find a direction\n\n**Difference of means.** Run a few hundred prompts that have the property and a few hundred that do not. Take the average activation at a chosen layer for each group and subtract. The result is a direction that separates them. It is crude, needs no training, and works far better than it has any right to.\n\n**A probe's weights.** The linear probe of the second lesson is a direction by construction.\n\n**A dictionary feature.** The decoder column for a sparse-autoencoder feature from the third lesson.\n\nAll three give a vector in the residual stream's space. Steering is the same whichever you used.\n\n## Doing it\n\n```python\n# d: direction at layer L, from difference of means, normalised\nd = (acts_pos[L].mean(0) - acts_neg[L].mean(0))\nd = d / d.norm()\n\ndef steer(resid, hook):\n    return resid + alpha * d          # add at every position\n\nout = model.generate(prompt, fwd_hooks=[(f\"blocks.{L}.hook_resid_post\", steer)])\n```\n\nOn a 2-billion-parameter open model in a free notebook this is about thirty lines including loading. Two knobs matter. The layer: middle layers usually steer best, because that is where abstract features live and there are enough layers after them to act on the change. The strength, `alpha`: too small and nothing happens; too large and the output dissolves into repetition. The usable range is a few multiples of the typical activation norm, found by trying five values.\n\nTry it on formal versus casual register, on Hindi versus English output, on sycophantic versus blunt feedback. Each works, imperfectly, and each is a direction the model was already using.\n\n## The refusal direction, in full\n\nModule six promised this. In 2024 researchers applied difference of means to thirteen open chat models: harmful instructions versus harmless ones, at a middle layer. Each model yielded a single direction. Subtracting it (projecting it out of the residual stream at every layer) produced a model that complied with almost any request while keeping its other capabilities. Adding it produced a model that refused to give a recipe for bread.\n\nFolding the projection into the weights makes it permanent, with no training at all. Such \"abliterated\" versions of most popular open models now exist publicly, made in an afternoon each.\n\nRead this carefully for what it does and does not say. It does not say safety training is useless; the direction only exists because training created it. It says that in these models refusal is **one feature**, thin and localisable, sitting on top of unchanged capability, and that every jailbreak in module six is an input that keeps that one direction from activating. Anyone deploying open weights should assume the feature can be removed, because it can, and put the actual safeguards where module four's tool lesson put them.\n\n## Persona vectors and monitoring\n\nIn 2025 the same recipe was applied to character traits (sycophancy, hostility, a tendency to invent) producing directions that predict when a model is drifting toward the trait. Two uses followed. Scan candidate training data by measuring how far it pushes along a trait direction, and drop what pushes the wrong way. And steer *against* a trait during fine-tuning, so the model does not learn it from data that would otherwise teach it: a kind of vaccination. Vectors for emotional tone, truthfulness, verbosity and output language have all been reported by the same method.\n\n## The limits\n\n**Side effects.** Directions are correlated. Steer toward honesty and the model gets terser; steer toward friendliness and it gets less accurate. You moved one direction and several features shared it, which is superposition again.\n\n**Brittleness.** The layer and strength are tuned per model. The direction found for one version does not transfer to the next; module six's drift applies to the model's internals as much as to its outputs.\n\n**It is not training.** Steering changes one forward pass and does nothing durable. For reliable behaviour change, training remains the tool; steering is for research, monitoring, quick experiments and, in a few reported cases, style control in production.\n\n**Closed models cannot be steered from outside.** You need the activations. This is one of the concrete capability differences between hosting an open model and calling an API, and it is why the interpretability results in this module are almost all on open weights.\n\n## What it shows\n\nSteering working at all is the strongest everyday evidence for the picture this module opened with. Features are directions; behaviour follows the direction; the model's dispositions are things you can locate and push on. That is the sentence to keep, and the next lesson asks whether the same is true of the model's facts."},"check":{"question":"Projecting one direction out of the residual stream at every layer of an open chat model makes it comply with almost any request while its other capabilities stay intact. What does this show about the refusal behaviour?","options":[{"text":"It is mediated by one feature direction rather than distributed across the model's knowledge.","correct":true},{"text":"It was never really trained into the weights, and the model was only refusing because of instructions in its hidden system prompt.","correct":false,"tempting":"The models in the study had no system prompt; the direction exists in the weights because safety training put it there, which is the opposite of it being absent."},{"text":"The model's safety comes from a separate classifier in the serving stack, and the projection disabled the connection to it.","correct":false,"tempting":"These were open weights run locally with no classifier present; the change was inside the model's own forward pass."},{"text":"Refusal is stored in the attention heads while factual knowledge is stored in the feed-forward layers, and projection removes the attention contribution while leaving the feed-forward knowledge untouched.","correct":false,"tempting":"Invents a division the study did not find; the projection was applied to the residual stream that both kinds of layer write into, and what it removed was one direction, not one kind of layer."}]},"prev":{"slug":"activation-patching-and-circuits","title":"Patching: finding which parts do the work","url":"https://addaly.com/learn/how-llms-work/activation-patching-and-circuits"},"next":{"slug":"where-facts-live","title":"Where a fact lives, and whether you can edit it","url":"https://addaly.com/learn/how-llms-work/where-facts-live"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Steering: pushing on a direction', in How a Language Model Actually Works. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/how-llms-work/steering-with-directions (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Steering: pushing on a direction","courseTitle":"How a Language Model Actually Works","module":{"index":8,"slug":"looking-inside","title":"Looking inside: what the weights actually hold"},"answer":null,"keyIdea":"Adding a feature’s direction to the residual stream changes behaviour in the way the feature’s meaning predicts, and the discovery that refusal in open chat models is one such direction (removable in an afternoon) is why the model’s own refusals cannot be the safety boundary.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-intervention-the-module-has-been-building-toward","runs":[{"text":"The intervention the module has been building toward"}]},{"t":"p","runs":[{"text":"If a feature is a direction, and if the model uses it, then adding a multiple of that direction to the residual stream during the forward pass should change the output in the way the feature’s meaning predicts. It does. This is activation steering, first demonstrated in 2023, and it is the cheapest experiment in this module with the largest implications."}]},{"t":"h2","id":"three-ways-to-find-a-direction","runs":[{"text":"Three ways to find a direction"}]},{"t":"p","runs":[{"text":"Difference of means.","b":true},{"text":" Run a few hundred prompts that have the property and a few hundred that do not. Take the average activation at a chosen layer for each group and subtract. The result is a direction that separates them. It is crude, needs no training, and works far better than it has any right to."}]},{"t":"p","runs":[{"text":"A probe’s weights.","b":true},{"text":" The linear probe of the second lesson is a direction by construction."}]},{"t":"p","runs":[{"text":"A dictionary feature.","b":true},{"text":" The decoder column for a sparse-autoencoder feature from the third lesson."}]},{"t":"p","runs":[{"text":"All three give a vector in the residual stream’s space. Steering is the same whichever you used."}]},{"t":"h2","id":"doing-it","runs":[{"text":"Doing it"}]},{"t":"code","lang":"python","label":"Python","text":"# d: direction at layer L, from difference of means, normalised\nd = (acts_pos[L].mean(0) - acts_neg[L].mean(0))\nd = d / d.norm()\n\ndef steer(resid, hook):\n    return resid + alpha * d          # add at every position\n\nout = model.generate(prompt, fwd_hooks=[(f\"blocks.{L}.hook_resid_post\", steer)])"},{"t":"p","runs":[{"text":"On a 2-billion-parameter open model in a free notebook this is about thirty lines including loading. Two knobs matter. The layer: middle layers usually steer best, because that is where abstract features live and there are enough layers after them to act on the change. The strength, "},{"text":"alpha","code":true},{"text":": too small and nothing happens; too large and the output dissolves into repetition. The usable range is a few multiples of the typical activation norm, found by trying five values."}]},{"t":"p","runs":[{"text":"Try it on formal versus casual register, on Hindi versus English output, on sycophantic versus blunt feedback. Each works, imperfectly, and each is a direction the model was already using."}]},{"t":"h2","id":"the-refusal-direction-in-full","runs":[{"text":"The refusal direction, in full"}]},{"t":"p","runs":[{"text":"Module six promised this. In 2024 researchers applied difference of means to thirteen open chat models: harmful instructions versus harmless ones, at a middle layer. Each model yielded a single direction. Subtracting it (projecting it out of the residual stream at every layer) produced a model that complied with almost any request while keeping its other capabilities. Adding it produced a model that refused to give a recipe for bread."}]},{"t":"p","runs":[{"text":"Folding the projection into the weights makes it permanent, with no training at all. Such “abliterated” versions of most popular open models now exist publicly, made in an afternoon each."}]},{"t":"p","runs":[{"text":"Read this carefully for what it does and does not say. It does not say safety training is useless; the direction only exists because training created it. It says that in these models refusal is "},{"text":"one feature","b":true},{"text":", thin and localisable, sitting on top of unchanged capability, and that every jailbreak in module six is an input that keeps that one direction from activating. Anyone deploying open weights should assume the feature can be removed, because it can, and put the actual safeguards where module four’s tool lesson put them."}]},{"t":"h2","id":"persona-vectors-and-monitoring","runs":[{"text":"Persona vectors and monitoring"}]},{"t":"p","runs":[{"text":"In 2025 the same recipe was applied to character traits (sycophancy, hostility, a tendency to invent) producing directions that predict when a model is drifting toward the trait. Two uses followed. Scan candidate training data by measuring how far it pushes along a trait direction, and drop what pushes the wrong way. And steer "},{"text":"against","i":true},{"text":" a trait during fine-tuning, so the model does not learn it from data that would otherwise teach it: a kind of vaccination. Vectors for emotional tone, truthfulness, verbosity and output language have all been reported by the same method."}]},{"t":"h2","id":"the-limits","runs":[{"text":"The limits"}],"role":"limit"},{"t":"p","runs":[{"text":"Side effects.","b":true},{"text":" Directions are correlated. Steer toward honesty and the model gets terser; steer toward friendliness and it gets less accurate. You moved one direction and several features shared it, which is superposition again."}]},{"t":"p","runs":[{"text":"Brittleness.","b":true},{"text":" The layer and strength are tuned per model. The direction found for one version does not transfer to the next; module six’s drift applies to the model’s internals as much as to its outputs."}]},{"t":"p","runs":[{"text":"It is not training.","b":true},{"text":" Steering changes one forward pass and does nothing durable. For reliable behaviour change, training remains the tool; steering is for research, monitoring, quick experiments and, in a few reported cases, style control in production."}]},{"t":"p","runs":[{"text":"Closed models cannot be steered from outside.","b":true},{"text":" You need the activations. This is one of the concrete capability differences between hosting an open model and calling an API, and it is why the interpretability results in this module are almost all on open weights."}]},{"t":"h2","id":"what-it-shows","runs":[{"text":"What it shows"}]},{"t":"p","runs":[{"text":"Steering working at all is the strongest everyday evidence for the picture this module opened with. Features are directions; behaviour follows the direction; the model’s dispositions are things you can locate and push on. That is the sentence to keep, and the next lesson asks whether the same is true of the model’s facts."}]}],"check":{"question":[{"text":"Projecting one direction out of the residual stream at every layer of an open chat model makes it comply with almost any request while its other capabilities stay intact. What does this show about the refusal behaviour?"}],"options":[{"text":[{"text":"It is mediated by one feature direction rather than distributed across the model’s knowledge."}],"correct":true,"tempting":null},{"text":[{"text":"It was never really trained into the weights, and the model was only refusing because of instructions in its hidden system prompt."}],"correct":false,"tempting":[{"text":"The models in the study had no system prompt; the direction exists in the weights because safety training put it there, which is the opposite of it being absent."}]},{"text":[{"text":"The model’s safety comes from a separate classifier in the serving stack, and the projection disabled the connection to it."}],"correct":false,"tempting":[{"text":"These were open weights run locally with no classifier present; the change was inside the model’s own forward pass."}]},{"text":[{"text":"Refusal is stored in the attention heads while factual knowledge is stored in the feed-forward layers, and projection removes the attention contribution while leaving the feed-forward knowledge untouched."}],"correct":false,"tempting":[{"text":"Invents a division the study did not find; the projection was applied to the residual stream that both kinds of layer write into, and what it removed was one direction, not one kind of layer."}]}],"order":[1,3,2,0],"why":null},"prev":{"slug":"activation-patching-and-circuits","title":"Patching: finding which parts do the work","minutes":9},"next":{"slug":"where-facts-live","title":"Where a fact lives, and whether you can edit it","minutes":9}}}