{"course":{"slug":"how-llms-work","title":"How a Language Model Actually Works","url":"https://addaly.com/learn/how-llms-work","level":"intermediate","track":"start"},"lesson":{"slug":"emergence-and-the-argument","title":"Emergent abilities, and the argument about whether they exist","url":"https://addaly.com/learn/how-llms-work/emergence-and-the-argument","position":48,"of":78,"minutes":8,"answer":"A capability that appears to switch on abruptly with scale may be a smoothly improving capability measured with an all-or-nothing metric, and the disagreement matters for anyone forecasting what models will do next.","keyIdea":"A capability that appears to switch on abruptly with scale may be a smoothly improving capability measured with an all-or-nothing metric, and the disagreement matters for anyone forecasting what models will do next."},"body":{"format":"markdown","text":"## The claim\n\nIn 2022 a widely read paper documented what it called emergent abilities: tasks on which models of a certain size score near zero, and models somewhat larger score well. Multi-step arithmetic, word unscrambling, transliteration. Plotted against scale, the curves look like a switch flipping.\n\nThe implication drawn was significant. If capabilities appear discontinuously and without warning, then you cannot predict what the next model will be able to do, and safety planning has to account for surprises.\n\n## The rebuttal\n\nIn 2023 a paper titled \"Are Emergent Abilities of Large Language Models a Mirage?\" argued that the discontinuity is often produced by the **metric**, not the model.\n\nThe argument is clean. Take multi-digit arithmetic scored by exact match: the whole answer is right or it is not. Suppose per-digit accuracy improves smoothly with scale: 80 per cent, then 90, then 95. The probability of getting all five digits right is that raised to the fifth power: 33 per cent, then 59, then 77. A smooth underlying improvement, passed through a threshold metric, produces a curve that looks like a jump.\n\nRe-score the same model outputs with a continuous metric (edit distance, per-token probability of the correct answer) and many of the sharp curves become smooth. The authors also showed they could *induce* apparent emergence in domains where nothing surprising was happening, by choosing a harsh metric.\n\n## Where the argument stands\n\nNot fully resolved, and the honest summary has three parts.\n\n**The metric critique is correct and important.** Many published emergence claims used exact-match or multiple-choice scoring and do soften under continuous metrics. Anyone reporting a capability jump should now be expected to show it under a graded metric.\n\n**It does not obviously explain everything.** Some capabilities remain hard to describe as smooth under any natural metric, and in-context learning in particular has an abruptness in training (associated with the formation of induction heads from module two) that looks like a genuine phase change in the mechanism rather than an artefact of scoring.\n\n**Both sides agree on the practical point.** Whether or not the underlying capability is smooth, *what a model can usefully do* can change quickly around a threshold. A model with 70 per cent per-step accuracy over a ten-step task succeeds 3 per cent of the time and is useless; at 95 per cent it succeeds 60 per cent of the time and is a product. The user experiences a discontinuity even if the parameter moved smoothly.\n\n## Why this matters to you\n\n**Reading claims.** When a paper or a launch reports a sudden capability, look at the metric. Exact match on a multi-step task will manufacture a jump from steady progress.\n\n**Evaluating your own system.** This is the transferable lesson. If your evaluation is pass-or-fail on a multi-step task, small real improvements will look like nothing until they suddenly look like everything, and you will misjudge the value of every change you make in between. Score the steps as well as the outcome. The evaluation course builds this properly. A ten-step task at 70 per cent per step has a 3 per cent end-to-end pass rate; the same task at 80 per cent per step passes 11 per cent of the time. The outcome metric calls that no progress. The per-step metric calls it what it is.\n\n**Forecasting.** Anyone confidently predicting the next model's capabilities is on shakier ground than they sound. Loss is predictable from scaling laws; the mapping from loss to capability is not, and this argument is precisely about how badly that mapping is understood.\n\n## The habit to take away\n\nWhen a system's measured performance jumps, ask two questions before concluding anything about the system. Did the underlying quantity change sharply, or did a threshold get crossed? And would a different metric have shown a straight line?\n\nThat is a generic scientific reflex rather than an AI one, which is why it is worth carrying: it applies to conversion rates, error budgets and student pass rates just as well as it applies here."},"check":{"question":"An agent's task-completion rate rises from 4 per cent to 55 per cent between two model versions, and the team concludes a new capability emerged. What should they check first?","options":[{"text":"Whether per-step accuracy improved smoothly, since a modest gain per step compounds into a large jump in all-or-nothing completion over a multi-step task.","correct":true},{"text":"Whether the benchmark was contaminated by the newer model's training data.","correct":false,"tempting":"Contamination is a genuine and common cause of sudden benchmark gains, so this is a well-founded thing to rule out second."},{"text":"Whether the newer model has a longer context window that fits the whole task.","correct":false,"tempting":"Context length does gate multi-step tasks, which makes it a sensible mechanical explanation for a threshold effect."},{"text":"Whether the evaluation harness changed between the two runs.","correct":false,"tempting":"Harness drift silently invalidates comparisons more often than anyone admits, so this is good practice, just not the first explanation here."}]},"prev":{"slug":"reinforcement-learning-on-checkable-answers","title":"Training against answers you can check","url":"https://addaly.com/learn/how-llms-work/reinforcement-learning-on-checkable-answers"},"next":{"slug":"hallucination-by-design","title":"Why hallucination is a property of the method","url":"https://addaly.com/learn/how-llms-work/hallucination-by-design"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Emergent abilities, and the argument about whether they exist', in How a Language Model Actually Works. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/how-llms-work/emergence-and-the-argument (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Emergent abilities, and the argument about whether they exist","courseTitle":"How a Language Model Actually Works","module":{"index":5,"slug":"where-the-weights-come-from","title":"Where the weights come from"},"answer":null,"keyIdea":"A capability that appears to switch on abruptly with scale may be a smoothly improving capability measured with an all-or-nothing metric, and the disagreement matters for anyone forecasting what models will do next.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-claim","runs":[{"text":"The claim"}]},{"t":"p","runs":[{"text":"In 2022 a widely read paper documented what it called emergent abilities: tasks on which models of a certain size score near zero, and models somewhat larger score well. Multi-step arithmetic, word unscrambling, transliteration. Plotted against scale, the curves look like a switch flipping."}]},{"t":"p","runs":[{"text":"The implication drawn was significant. If capabilities appear discontinuously and without warning, then you cannot predict what the next model will be able to do, and safety planning has to account for surprises."}]},{"t":"h2","id":"the-rebuttal","runs":[{"text":"The rebuttal"}]},{"t":"p","runs":[{"text":"In 2023 a paper titled “Are Emergent Abilities of Large Language Models a Mirage?” argued that the discontinuity is often produced by the "},{"text":"metric","b":true},{"text":", not the model."}]},{"t":"p","runs":[{"text":"The argument is clean. Take multi-digit arithmetic scored by exact match: the whole answer is right or it is not. Suppose per-digit accuracy improves smoothly with scale: 80 per cent, then 90, then 95. The probability of getting all five digits right is that raised to the fifth power: 33 per cent, then 59, then 77. A smooth underlying improvement, passed through a threshold metric, produces a curve that looks like a jump."}]},{"t":"p","runs":[{"text":"Re-score the same model outputs with a continuous metric (edit distance, per-token probability of the correct answer) and many of the sharp curves become smooth. The authors also showed they could "},{"text":"induce","i":true},{"text":" apparent emergence in domains where nothing surprising was happening, by choosing a harsh metric."}]},{"t":"h2","id":"where-the-argument-stands","runs":[{"text":"Where the argument stands"}]},{"t":"p","runs":[{"text":"Not fully resolved, and the honest summary has three parts."}]},{"t":"p","runs":[{"text":"The metric critique is correct and important.","b":true},{"text":" Many published emergence claims used exact-match or multiple-choice scoring and do soften under continuous metrics. Anyone reporting a capability jump should now be expected to show it under a graded metric."}]},{"t":"p","runs":[{"text":"It does not obviously explain everything.","b":true},{"text":" Some capabilities remain hard to describe as smooth under any natural metric, and in-context learning in particular has an abruptness in training (associated with the formation of induction heads from module two) that looks like a genuine phase change in the mechanism rather than an artefact of scoring."}]},{"t":"p","runs":[{"text":"Both sides agree on the practical point.","b":true},{"text":" Whether or not the underlying capability is smooth, "},{"text":"what a model can usefully do","i":true},{"text":" can change quickly around a threshold. A model with 70 per cent per-step accuracy over a ten-step task succeeds 3 per cent of the time and is useless; at 95 per cent it succeeds 60 per cent of the time and is a product. The user experiences a discontinuity even if the parameter moved smoothly."}]},{"t":"h2","id":"why-this-matters-to-you","runs":[{"text":"Why this matters to you"}]},{"t":"p","runs":[{"text":"Reading claims.","b":true},{"text":" When a paper or a launch reports a sudden capability, look at the metric. Exact match on a multi-step task will manufacture a jump from steady progress."}]},{"t":"p","runs":[{"text":"Evaluating your own system.","b":true},{"text":" This is the transferable lesson. If your evaluation is pass-or-fail on a multi-step task, small real improvements will look like nothing until they suddenly look like everything, and you will misjudge the value of every change you make in between. Score the steps as well as the outcome. The evaluation course builds this properly. A ten-step task at 70 per cent per step has a 3 per cent end-to-end pass rate; the same task at 80 per cent per step passes 11 per cent of the time. The outcome metric calls that no progress. The per-step metric calls it what it is."}]},{"t":"p","runs":[{"text":"Forecasting.","b":true},{"text":" Anyone confidently predicting the next model’s capabilities is on shakier ground than they sound. Loss is predictable from scaling laws; the mapping from loss to capability is not, and this argument is precisely about how badly that mapping is understood."}]},{"t":"h2","id":"the-habit-to-take-away","runs":[{"text":"The habit to take away"}]},{"t":"p","runs":[{"text":"When a system’s measured performance jumps, ask two questions before concluding anything about the system. Did the underlying quantity change sharply, or did a threshold get crossed? And would a different metric have shown a straight line?"}]},{"t":"p","runs":[{"text":"That is a generic scientific reflex rather than an AI one, which is why it is worth carrying: it applies to conversion rates, error budgets and student pass rates just as well as it applies here."}]}],"check":{"question":[{"text":"An agent’s task-completion rate rises from 4 per cent to 55 per cent between two model versions, and the team concludes a new capability emerged. What should they check first?"}],"options":[{"text":[{"text":"Whether per-step accuracy improved smoothly, since a modest gain per step compounds into a large jump in all-or-nothing completion over a multi-step task."}],"correct":true,"tempting":null},{"text":[{"text":"Whether the benchmark was contaminated by the newer model’s training data."}],"correct":false,"tempting":[{"text":"Contamination is a genuine and common cause of sudden benchmark gains, so this is a well-founded thing to rule out second."}]},{"text":[{"text":"Whether the newer model has a longer context window that fits the whole task."}],"correct":false,"tempting":[{"text":"Context length does gate multi-step tasks, which makes it a sensible mechanical explanation for a threshold effect."}]},{"text":[{"text":"Whether the evaluation harness changed between the two runs."}],"correct":false,"tempting":[{"text":"Harness drift silently invalidates comparisons more often than anyone admits, so this is good practice, just not the first explanation here."}]}],"order":[1,0,2,3],"why":null},"prev":{"slug":"reinforcement-learning-on-checkable-answers","title":"Training against answers you can check","minutes":9},"next":{"slug":"hallucination-by-design","title":"Why hallucination is a property of the method","minutes":8}}}