Emergent abilities, and the argument about whether they exist

The claim

In 2022 a widely read paper documented what it called emergent abilities: tasks on which models of a certain size score near zero, and models somewhat larger score well. Multi-step arithmetic, word unscrambling, transliteration. Plotted against scale, the curves look like a switch flipping.

The implication drawn was significant. If capabilities appear discontinuously and without warning, then you cannot predict what the next model will be able to do, and safety planning has to account for surprises.

The rebuttal

In 2023 a paper titled “Are Emergent Abilities of Large Language Models a Mirage?” argued that the discontinuity is often produced by the metric, not the model.

The argument is clean. Take multi-digit arithmetic scored by exact match: the whole answer is right or it is not. Suppose per-digit accuracy improves smoothly with scale: 80 per cent, then 90, then 95. The probability of getting all five digits right is that raised to the fifth power: 33 per cent, then 59, then 77. A smooth underlying improvement, passed through a threshold metric, produces a curve that looks like a jump.

Re-score the same model outputs with a continuous metric (edit distance, per-token probability of the correct answer) and many of the sharp curves become smooth. The authors also showed they could induce apparent emergence in domains where nothing surprising was happening, by choosing a harsh metric.

Where the argument stands

Not fully resolved, and the honest summary has three parts.

The metric critique is correct and important. Many published emergence claims used exact-match or multiple-choice scoring and do soften under continuous metrics. Anyone reporting a capability jump should now be expected to show it under a graded metric.

It does not obviously explain everything. Some capabilities remain hard to describe as smooth under any natural metric, and in-context learning in particular has an abruptness in training (associated with the formation of induction heads from module two) that looks like a genuine phase change in the mechanism rather than an artefact of scoring.

Both sides agree on the practical point. Whether or not the underlying capability is smooth, what a model can usefully do can change quickly around a threshold. A model with 70 per cent per-step accuracy over a ten-step task succeeds 3 per cent of the time and is useless; at 95 per cent it succeeds 60 per cent of the time and is a product. The user experiences a discontinuity even if the parameter moved smoothly.

Why this matters to you

Reading claims. When a paper or a launch reports a sudden capability, look at the metric. Exact match on a multi-step task will manufacture a jump from steady progress.

Evaluating your own system. This is the transferable lesson. If your evaluation is pass-or-fail on a multi-step task, small real improvements will look like nothing until they suddenly look like everything, and you will misjudge the value of every change you make in between. Score the steps as well as the outcome. The evaluation course builds this properly. A ten-step task at 70 per cent per step has a 3 per cent end-to-end pass rate; the same task at 80 per cent per step passes 11 per cent of the time. The outcome metric calls that no progress. The per-step metric calls it what it is.

Forecasting. Anyone confidently predicting the next model’s capabilities is on shakier ground than they sound. Loss is predictable from scaling laws; the mapping from loss to capability is not, and this argument is precisely about how badly that mapping is understood.

The habit to take away

When a system’s measured performance jumps, ask two questions before concluding anything about the system. Did the underlying quantity change sharply, or did a threshold get crossed? And would a different metric have shown a straight line?

That is a generic scientific reflex rather than an AI one, which is why it is worth carrying: it applies to conversion rates, error budgets and student pass rates just as well as it applies here.