Unlearning: why removing something is harder than adding it

The request

“Remove this person’s data from the model.” “Remove this book.” “Remove this capability.” Each sounds like deleting a row, and the last two lessons explain why it is not. A fact is one lookup among many redundant ones. A memorised passage is a path through weights shared with everything else the model knows. Nothing is stored in a place. There is no row.

Unlearning is the research programme that tries to honour such requests anyway, and its results are the clearest available evidence about how knowledge sits in a model.

The methods

Gradient ascent on the forget set. Training pushes loss down on text; unlearning pushes it up on the target. It removes verbatim recall effectively and overshoots easily: push too hard and the model produces gibberish on anything nearby, because the paths being damaged are shared. A retain set of ordinary text is trained on at the same time to limit the collateral.

Teaching refusal. Train the model to answer “I don’t know” or a neutral alternative on the target, with a preference-style loss that avoids the instability of pure ascent. Safer, and it does what it says: the model learns to decline the topic. Whether the knowledge went anywhere is the question the rest of this lesson answers.

Scrambling the representation. A 2024 method perturbs the model’s internal representations for a target domain at a middle layer (the published case was hazardous biological knowledge) while holding other representations in place. On its benchmark it removes the capability while preserving general performance, and it is the strongest of the current family.

Weight editing. The previous lesson’s method, aimed at deletion, with the previous lesson’s fragilities.

The result that matters

Take a model that has been unlearned by any of the above and that fails every ordinary prompt about the target. Fine-tune it briefly on a small amount of data: a few hundred examples on the topic, or in several studies a few hundred examples on something unrelated and benign. The knowledge comes back.

In one 2024 study, quantising an unlearned model to four bits restored much of what had been removed: the unlearning lived in small weight adjustments that the rounding erased, while the original knowledge lived in larger ones that survived.

Module six’s jailbreaks sometimes recover unlearned content too, by the same mechanism as they recover refused content: the direction that suppresses the answer is one feature, and an input can keep it from firing.

The interpretation is hard to avoid. Most current methods suppress access rather than erase content. The lookup is still there, disconnected from the prompt patterns that used to reach it, and reconnection is cheap because the model’s general fine-tunability is exactly what makes it useful.

Evaluating an unlearning claim

Three questions, and a published claim that answers only the first two is not a claim of unlearning.

1. Forget quality. Does the target fail under ordinary prompts, and under adversarial ones, under a probe of the second lesson, under the extraction test of the previous lesson?
2. Retain quality. Did general ability survive? Related knowledge? Fluency?
3. Robustness. Does it stay gone after a little fine-tuning, after quantisation, after a jailbreak?

Public benchmarks exist for each. One uses invented authors, so the ground truth of what was ever known is exact. Others target hazardous knowledge, or copyrighted books. Most published methods score well on the first two questions and poorly on the third, and the gap between them is the entire field.

The legal edge

Data-protection regimes with a right to erasure (the European rules, India’s 2023 Act, others) were written for databases, where a row can be found and deleted. How they apply to weights is unsettled, differs by jurisdiction, and regulators have issued guidance that acknowledges the technical difficulty without resolving it. This lesson gives no advice on it. What it can say is that the technical question underneath (whether personal data in a model can be removed rather than hidden) currently has the answer above, and any process that depends on the stronger answer is depending on something nobody has demonstrated.

What actually works

For personal data, prevention: filter before training, and keep personal data out of the weights altogether by holding it in a store the model retrieves from (module four’s architecture) which is a store you can delete from. For a model already trained on something it should not have seen, retraining from filtered data is the only method that provably removes it; everything else is suppression with a measured relearning cost. For capability removal in a safety context, the representation-level methods are the best available, alongside the acknowledgement that open weights can be fine-tuned by whoever holds them.

The honest state

Active, fast-moving research. Methods improve on the benchmarks each year and the robustness gap has narrowed a little without closing. Until it closes, read “the model has unlearned X” as “the model no longer says X until someone fine-tunes it for an hour”, and design so that the things you would need to remove were never in the weights to begin with.