{"course":{"slug":"prompting","title":"Getting Real Work out of a Model","url":"https://addaly.com/learn/prompting","level":"beginner","track":"start"},"lesson":{"slug":"make-it-show-its-work","title":"Make it show its work","url":"https://addaly.com/learn/prompting/make-it-show-its-work","position":37,"of":72,"minutes":8,"answer":"Shown reasoning is not a log of how the answer was made, but it puts claims where you can check them.","keyIdea":"Shown reasoning is not a log of how the answer was made, but it puts claims where you can check them."},"body":{"format":"markdown","text":"## The scheduling problem\n\n> When should we hold the team call? People are in Lagos, Berlin and Manila. Everyone works 9 to 6 local time.\n\nYou will often get a confident answer that does not survive checking. Manila is seven hours ahead of Lagos; the overlap is narrow and easy to get wrong in one pass.\n\nNow ask for the steps:\n\n> First convert each person's 9 to 6 into UTC and show the three ranges. Then list every UTC hour where all three overlap. Then pick one and say why. If there is no hour where all three overlap, say so rather than choosing the least bad option.\n\nThis is usually right, and more importantly you can see where it went wrong when it is not. If the Berlin conversion is off by an hour, you can spot that in two seconds. The single-answer version gives you nothing to check.\n\n## Why intermediate steps help\n\nFor tasks with several dependent stages (conversions, comparisons against a list of constraints, multi-step arithmetic, working through a document clause by clause) writing the steps out improves accuracy. The intermediate text is where the work gets done. Skip it and you are asking for the last line of a calculation that was never performed.\n\nIt helps much less for recall, tone, translation, or summarising a short passage. There, asking for steps mostly makes the answer longer. \"Think step by step\" is not a universal upgrade, and adding it everywhere is a habit worth dropping.\n\nA lot of newer products do this reasoning internally before answering, so you may not need to ask at all. What you can still ask for is the part you want to inspect: \"list the constraints you are checking against, then answer.\"\n\n## The caveat that matters most\n\nThe steps a model shows you are not a log of how it produced the answer. They are text generated alongside the answer, in the shape of an explanation.\n\nThis is well studied and the finding is uncomfortable: models sometimes reach the right answer through stated steps containing an error, and sometimes produce flawless-looking reasoning to a wrong conclusion. The written reasoning can also be shaped by hints in your prompt that never get mentioned in the steps.\n\nSo do not treat shown reasoning as proof. Treat it as material you can check. Its value is that it puts intermediate claims where your eyes can reach them: the UTC conversions, the clause it thinks says X, the figure it took from the table. Those are checkable in seconds, and checking them is the point.\n\nA wrong step under a right answer is not a paradox to explain away. It means both need looking at.\n\n## Give it permission to fail\n\nNotice the last line of the good prompt: *if there is no hour where all three overlap, say so rather than choosing the least bad option.*\n\nWithout that, a model will usually produce an answer, because producing an answer is what the shape of the conversation calls for. Naming the escape route makes \"there is no valid answer\" an available response instead of a conversational failure.\n\nUse this constantly:\n\n- \"If the contract does not cover this, say so.\"\n- \"If none of these three candidates meets the requirement, say none.\"\n- \"If you need a fact I have not given you, ask me instead of assuming.\"\n\nThat last one turns a guess into a question, which is nearly always the better trade.\n\n## What to ask for in practice\n\nFor a task with constraints: \"list the constraints, then check each option against each constraint, then conclude.\"\n\nFor a document task: \"quote the sentence you are relying on, then interpret it.\" A quote is verifiable. An interpretation on its own is not.\n\nFor anything numerical: \"show the numbers you used and where each came from.\"\n\nAll three convert an opinion you have to trust into work you can audit."},"check":{"question":"A model shows five reasoning steps and reaches the correct answer. You read the steps and find that step 3 is clearly wrong, yet the final answer is still right. What should you conclude?","options":[{"text":"The final answer is probably wrong too, since a chain of reasoning is only as strong as its weakest link.","correct":false,"tempting":"It treats the written steps as a genuine derivation, so ordinary logic about chains gets applied to text that was not actually a chain."},{"text":"The written steps are an explanation produced alongside the answer, not a record of how it was reached, so a wrong step above a right answer is a known pattern, and both still need checking.","correct":true},{"text":"The model must have caught and fixed the error silently, so step 3 is a slip in the write-up and can be ignored.","correct":false,"tempting":"We assume a coherent author behind coherent prose, so a contradiction gets read as a typo rather than as evidence about how the text was produced."},{"text":"Chain of thought is unreliable and should not be requested at all.","correct":false,"tempting":"A fair overcorrection once you learn the steps can be unfaithful, but they still improve accuracy on multi-step work and give you something concrete to verify."}]},"prev":{"slug":"output-that-fails-visibly","title":"Building the failure into the format","url":"https://addaly.com/learn/prompting/output-that-fails-visibly"},"next":{"slug":"when-steps-help","title":"Which tasks get better with steps, and which do not","url":"https://addaly.com/learn/prompting/when-steps-help"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Make it show its work', in Getting Real Work out of a Model. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/prompting/make-it-show-its-work (Accessed: 31 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Make it show its work","courseTitle":"Getting Real Work out of a Model","module":{"index":5,"slug":"making-it-work-the-problem","title":"Making it work the problem"},"answer":null,"keyIdea":"Shown reasoning is not a log of how the answer was made, but it puts claims where you can check them.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-scheduling-problem","runs":[{"text":"The scheduling problem"}]},{"t":"quote","runs":[{"text":"When should we hold the team call? People are in Lagos, Berlin and Manila. Everyone works 9 to 6 local time."}]},{"t":"p","runs":[{"text":"You will often get a confident answer that does not survive checking. Manila is seven hours ahead of Lagos; the overlap is narrow and easy to get wrong in one pass."}]},{"t":"p","runs":[{"text":"Now ask for the steps:"}]},{"t":"quote","runs":[{"text":"First convert each person’s 9 to 6 into UTC and show the three ranges. Then list every UTC hour where all three overlap. Then pick one and say why. If there is no hour where all three overlap, say so rather than choosing the least bad option."}]},{"t":"p","runs":[{"text":"This is usually right, and more importantly you can see where it went wrong when it is not. If the Berlin conversion is off by an hour, you can spot that in two seconds. The single-answer version gives you nothing to check."}]},{"t":"h2","id":"why-intermediate-steps-help","runs":[{"text":"Why intermediate steps help"}]},{"t":"p","runs":[{"text":"For tasks with several dependent stages (conversions, comparisons against a list of constraints, multi-step arithmetic, working through a document clause by clause) writing the steps out improves accuracy. The intermediate text is where the work gets done. Skip it and you are asking for the last line of a calculation that was never performed."}]},{"t":"p","runs":[{"text":"It helps much less for recall, tone, translation, or summarising a short passage. There, asking for steps mostly makes the answer longer. “Think step by step” is not a universal upgrade, and adding it everywhere is a habit worth dropping."}]},{"t":"p","runs":[{"text":"A lot of newer products do this reasoning internally before answering, so you may not need to ask at all. What you can still ask for is the part you want to inspect: “list the constraints you are checking against, then answer.”"}]},{"t":"h2","id":"the-caveat-that-matters-most","runs":[{"text":"The caveat that matters most"}]},{"t":"p","runs":[{"text":"The steps a model shows you are not a log of how it produced the answer. They are text generated alongside the answer, in the shape of an explanation."}]},{"t":"p","runs":[{"text":"This is well studied and the finding is uncomfortable: models sometimes reach the right answer through stated steps containing an error, and sometimes produce flawless-looking reasoning to a wrong conclusion. The written reasoning can also be shaped by hints in your prompt that never get mentioned in the steps."}]},{"t":"p","runs":[{"text":"So do not treat shown reasoning as proof. Treat it as material you can check. Its value is that it puts intermediate claims where your eyes can reach them: the UTC conversions, the clause it thinks says X, the figure it took from the table. Those are checkable in seconds, and checking them is the point."}]},{"t":"p","runs":[{"text":"A wrong step under a right answer is not a paradox to explain away. It means both need looking at."}]},{"t":"h2","id":"give-it-permission-to-fail","runs":[{"text":"Give it permission to fail"}]},{"t":"p","runs":[{"text":"Notice the last line of the good prompt: "},{"text":"if there is no hour where all three overlap, say so rather than choosing the least bad option.","i":true}]},{"t":"p","runs":[{"text":"Without that, a model will usually produce an answer, because producing an answer is what the shape of the conversation calls for. Naming the escape route makes “there is no valid answer” an available response instead of a conversational failure."}]},{"t":"p","runs":[{"text":"Use this constantly:"}]},{"t":"ul","items":[[{"text":"“If the contract does not cover this, say so.”"}],[{"text":"“If none of these three candidates meets the requirement, say none.”"}],[{"text":"“If you need a fact I have not given you, ask me instead of assuming.”"}]]},{"t":"p","runs":[{"text":"That last one turns a guess into a question, which is nearly always the better trade."}]},{"t":"h2","id":"what-to-ask-for-in-practice","runs":[{"text":"What to ask for in practice"}]},{"t":"p","runs":[{"text":"For a task with constraints: “list the constraints, then check each option against each constraint, then conclude.”"}]},{"t":"p","runs":[{"text":"For a document task: “quote the sentence you are relying on, then interpret it.” A quote is verifiable. An interpretation on its own is not."}]},{"t":"p","runs":[{"text":"For anything numerical: “show the numbers you used and where each came from.”"}]},{"t":"p","runs":[{"text":"All three convert an opinion you have to trust into work you can audit."}]}],"check":{"question":[{"text":"A model shows five reasoning steps and reaches the correct answer. You read the steps and find that step 3 is clearly wrong, yet the final answer is still right. What should you conclude?"}],"options":[{"text":[{"text":"The final answer is probably wrong too, since a chain of reasoning is only as strong as its weakest link."}],"correct":false,"tempting":[{"text":"It treats the written steps as a genuine derivation, so ordinary logic about chains gets applied to text that was not actually a chain."}]},{"text":[{"text":"The written steps are an explanation produced alongside the answer, not a record of how it was reached, so a wrong step above a right answer is a known pattern, and both still need checking."}],"correct":true,"tempting":null},{"text":[{"text":"The model must have caught and fixed the error silently, so step 3 is a slip in the write-up and can be ignored."}],"correct":false,"tempting":[{"text":"We assume a coherent author behind coherent prose, so a contradiction gets read as a typo rather than as evidence about how the text was produced."}]},{"text":[{"text":"Chain of thought is unreliable and should not be requested at all."}],"correct":false,"tempting":[{"text":"A fair overcorrection once you learn the steps can be unfaithful, but they still improve accuracy on multi-step work and give you something concrete to verify."}]}],"order":[3,0,2,1],"why":null},"prev":{"slug":"output-that-fails-visibly","title":"Building the failure into the format","minutes":8},"next":{"slug":"when-steps-help","title":"Which tasks get better with steps, and which do not","minutes":8}}}