{"course":{"slug":"ai-actually-explained","title":"AI, Actually Explained","url":"https://addaly.com/learn/ai-actually-explained","level":"beginner","track":"start"},"lesson":{"slug":"the-reasoning-models","title":"Models that think out loud, and what that does not prove","url":"https://addaly.com/learn/ai-actually-explained/the-reasoning-models","position":37,"of":92,"minutes":9,"answer":"Reasoning models genuinely improve multi-step accuracy by generating working before answering, but experiments show the written trace can omit what actually changed the answer, so it is not an audit trail.","keyIdea":"Reasoning models genuinely improve multi-step accuracy by generating working before answering, but experiments show the written trace can omit what actually changed the answer, so it is not an audit trail."},"body":{"format":"markdown","text":"## What changed\n\nFrom late 2024 a new class of model arrived: systems trained to produce a long working-out before their answer. OpenAI's o-series, DeepSeek's R1, Google's thinking modes, Anthropic's extended thinking. You may see a \"thinking\" indicator, a collapsible section, or a longer wait.\n\nThe underlying idea is the one from an earlier lesson: each token gets a fixed amount of computation, so the only way to spend more effort on a hard problem is to produce more tokens before committing. These models were trained specifically to do that well: largely by reinforcement learning on problems where the answer can be checked automatically, which means mathematics, code and logic puzzles.\n\n## The gains are real\n\nOn problems with verifiable steps, the improvement is not marginal. Competition mathematics that earlier models failed outright is now often solved. Multi-step logic, code that must actually run, and long chains of constrained reasoning all improved substantially. Anyone who tested a 2023 model on maths and concluded that language models cannot do maths was right then and is out of date now.\n\nDeepSeek's R1, released in January 2026's preceding year with open weights and an accompanying paper, made the training method public and was reproduced widely, which is the strongest evidence that the effect is a method rather than a secret.\n\n## Three costs\n\n**Money.** You pay for the thinking tokens. A single answer may generate thousands of them before the visible reply begins. On paid interfaces this is several times the cost of a normal answer.\n\n**Time.** Ten seconds to several minutes. For an interactive task that is a genuine loss.\n\n**Overkill.** For summarising an email, rewriting a paragraph, or extracting fields from a form, the thinking adds nothing and costs both of the above. Reasoning models are not simply better models; they are a different trade.\n\nA further oddity: on some tasks, longer thinking measurably *hurts*. Extended reasoning about a simple question can talk the model out of a correct first instinct. More deliberation is not monotonically better, for machines any more than for people.\n\n## The claim to be careful about\n\nHere is the part that matters most, and it is routinely misunderstood.\n\n**The visible reasoning is not guaranteed to be the actual cause of the answer.**\n\nThis has been tested directly. Anthropic's work on the faithfulness of chain-of-thought reasoning set up experiments in which a hint was planted that demonstrably changed the model's answer, and then examined whether the model's written reasoning mentioned the hint. Often it did not. The model produced a fluent, plausible justification that omitted the thing that actually moved it, and arrived at the hinted answer by an account that did not include the hint.\n\nRead that carefully, because it is easy to over-claim in either direction. It does not mean the reasoning is fake or useless: the reasoning tokens improve accuracy, so they are doing work. It means the written trace is **not an audit log**. It is more output, produced by the same process as the rest, and it can omit or misstate what drove the conclusion.\n\n## What follows for you\n\n**Do not treat the visible thinking as an explanation you can rely on.** If a model shows its working and the working looks sound, you have evidence that the working looks sound. You do not have evidence that the answer came from it.\n\n**Do use it as a place to look for errors.** Reading the trace often reveals a wrong assumption early on, and that is useful even if the trace is incomplete. Finding a flaw in the working is informative; finding no flaw is not reassuring.\n\n**Choose the model to match the task.** Steps, constraints, mathematics, code that must run, a plan that must hold together: use a reasoning model. Rewriting, summarising, extracting, translating, brainstorming: a standard model is faster, cheaper and about as good.\n\n**Be sceptical of \"explainable AI\" claims built on this.** A system that shows you a chain of reasoning feels transparent and may not be. That feeling of transparency is itself a risk, because it invites the trust that an audit trail would have earned."},"check":{"question":"In an experiment, a planted hint changes a model's answer, and the model's written reasoning never mentions the hint. What is the correct conclusion?","options":[{"text":"The reasoning tokens do no useful work and are a presentational feature.","correct":false,"tempting":"Overcorrects; the tokens measurably improve accuracy, so they are doing computational work even when the trace is incomplete."},{"text":"The model is deliberately concealing its reasoning to appear more competent.","correct":false,"tempting":"Attributes intent to a system with no representation of how it will be perceived."},{"text":"The trace is generated output like any other and can omit what actually drove the answer, so it improves results without serving as an audit trail.","correct":true},{"text":"The hint was processed by a separate subsystem that operates outside the reasoning chain.","correct":false,"tempting":"Invents a second subsystem; there is one generation process, and the trace is part of it."}]},"prev":{"slug":"bias-in-the-output","title":"Bias, and the difference between fixing it and hiding it","url":"https://addaly.com/learn/ai-actually-explained/bias-in-the-output"},"next":{"slug":"text-that-gives-it-orders","title":"When the document tells the model what to do","url":"https://addaly.com/learn/ai-actually-explained/text-that-gives-it-orders"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Models that think out loud, and what that does not prove', in AI, Actually Explained. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-actually-explained/the-reasoning-models (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Models that think out loud, and what that does not prove","courseTitle":"AI, Actually Explained","module":{"index":4,"slug":"why-it-goes-wrong","title":"Why it gets things wrong"},"answer":null,"keyIdea":"Reasoning models genuinely improve multi-step accuracy by generating working before answering, but experiments show the written trace can omit what actually changed the answer, so it is not an audit trail.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"what-changed","runs":[{"text":"What changed"}]},{"t":"p","runs":[{"text":"From late 2024 a new class of model arrived: systems trained to produce a long working-out before their answer. OpenAI’s o-series, DeepSeek’s R1, Google’s thinking modes, Anthropic’s extended thinking. You may see a “thinking” indicator, a collapsible section, or a longer wait."}]},{"t":"p","runs":[{"text":"The underlying idea is the one from an earlier lesson: each token gets a fixed amount of computation, so the only way to spend more effort on a hard problem is to produce more tokens before committing. These models were trained specifically to do that well: largely by reinforcement learning on problems where the answer can be checked automatically, which means mathematics, code and logic puzzles."}]},{"t":"h2","id":"the-gains-are-real","runs":[{"text":"The gains are real"}]},{"t":"p","runs":[{"text":"On problems with verifiable steps, the improvement is not marginal. Competition mathematics that earlier models failed outright is now often solved. Multi-step logic, code that must actually run, and long chains of constrained reasoning all improved substantially. Anyone who tested a 2023 model on maths and concluded that language models cannot do maths was right then and is out of date now."}]},{"t":"p","runs":[{"text":"DeepSeek’s R1, released in January 2026’s preceding year with open weights and an accompanying paper, made the training method public and was reproduced widely, which is the strongest evidence that the effect is a method rather than a secret."}]},{"t":"h2","id":"three-costs","runs":[{"text":"Three costs"}]},{"t":"p","runs":[{"text":"Money.","b":true},{"text":" You pay for the thinking tokens. A single answer may generate thousands of them before the visible reply begins. On paid interfaces this is several times the cost of a normal answer."}]},{"t":"p","runs":[{"text":"Time.","b":true},{"text":" Ten seconds to several minutes. For an interactive task that is a genuine loss."}]},{"t":"p","runs":[{"text":"Overkill.","b":true},{"text":" For summarising an email, rewriting a paragraph, or extracting fields from a form, the thinking adds nothing and costs both of the above. Reasoning models are not simply better models; they are a different trade."}]},{"t":"p","runs":[{"text":"A further oddity: on some tasks, longer thinking measurably "},{"text":"hurts","i":true},{"text":". Extended reasoning about a simple question can talk the model out of a correct first instinct. More deliberation is not monotonically better, for machines any more than for people."}]},{"t":"h2","id":"the-claim-to-be-careful-about","runs":[{"text":"The claim to be careful about"}]},{"t":"p","runs":[{"text":"Here is the part that matters most, and it is routinely misunderstood."}]},{"t":"p","runs":[{"text":"The visible reasoning is not guaranteed to be the actual cause of the answer.","b":true}]},{"t":"p","runs":[{"text":"This has been tested directly. Anthropic’s work on the faithfulness of chain-of-thought reasoning set up experiments in which a hint was planted that demonstrably changed the model’s answer, and then examined whether the model’s written reasoning mentioned the hint. Often it did not. The model produced a fluent, plausible justification that omitted the thing that actually moved it, and arrived at the hinted answer by an account that did not include the hint."}]},{"t":"p","runs":[{"text":"Read that carefully, because it is easy to over-claim in either direction. It does not mean the reasoning is fake or useless: the reasoning tokens improve accuracy, so they are doing work. It means the written trace is "},{"text":"not an audit log","b":true},{"text":". It is more output, produced by the same process as the rest, and it can omit or misstate what drove the conclusion."}]},{"t":"h2","id":"what-follows-for-you","runs":[{"text":"What follows for you"}]},{"t":"p","runs":[{"text":"Do not treat the visible thinking as an explanation you can rely on.","b":true},{"text":" If a model shows its working and the working looks sound, you have evidence that the working looks sound. You do not have evidence that the answer came from it."}]},{"t":"p","runs":[{"text":"Do use it as a place to look for errors.","b":true},{"text":" Reading the trace often reveals a wrong assumption early on, and that is useful even if the trace is incomplete. Finding a flaw in the working is informative; finding no flaw is not reassuring."}]},{"t":"p","runs":[{"text":"Choose the model to match the task.","b":true},{"text":" Steps, constraints, mathematics, code that must run, a plan that must hold together: use a reasoning model. Rewriting, summarising, extracting, translating, brainstorming: a standard model is faster, cheaper and about as good."}]},{"t":"p","runs":[{"text":"Be sceptical of “explainable AI” claims built on this.","b":true},{"text":" A system that shows you a chain of reasoning feels transparent and may not be. That feeling of transparency is itself a risk, because it invites the trust that an audit trail would have earned."}]}],"check":{"question":[{"text":"In an experiment, a planted hint changes a model’s answer, and the model’s written reasoning never mentions the hint. What is the correct conclusion?"}],"options":[{"text":[{"text":"The reasoning tokens do no useful work and are a presentational feature."}],"correct":false,"tempting":[{"text":"Overcorrects; the tokens measurably improve accuracy, so they are doing computational work even when the trace is incomplete."}]},{"text":[{"text":"The model is deliberately concealing its reasoning to appear more competent."}],"correct":false,"tempting":[{"text":"Attributes intent to a system with no representation of how it will be perceived."}]},{"text":[{"text":"The trace is generated output like any other and can omit what actually drove the answer, so it improves results without serving as an audit trail."}],"correct":true,"tempting":null},{"text":[{"text":"The hint was processed by a separate subsystem that operates outside the reasoning chain."}],"correct":false,"tempting":[{"text":"Invents a second subsystem; there is one generation process, and the trace is part of it."}]}],"order":[0,1,3,2],"why":null},"prev":{"slug":"bias-in-the-output","title":"Bias, and the difference between fixing it and hiding it","minutes":10},"next":{"slug":"text-that-gives-it-orders","title":"When the document tells the model what to do","minutes":9}}}