{"course":{"slug":"prompting","title":"Getting Real Work out of a Model","url":"https://addaly.com/learn/prompting","level":"beginner","track":"start"},"lesson":{"slug":"how-much-checking","title":"How much checking is enough","url":"https://addaly.com/learn/prompting/how-much-checking","position":63,"of":72,"minutes":8,"answer":"Sort by what happens if the answer is wrong rather than by how confident it sounds, and design the prompt so the checking is a text search rather than a re-reading.","keyIdea":"Sort by what happens if the answer is wrong rather than by how confident it sounds, and design the prompt so the checking is a text search rather than a re-reading."},"body":{"format":"markdown","text":"## Checking everything is the same as checking nothing\n\nNobody verifies every claim in every answer. People who say they do are describing an intention. What actually happens is that effort spread evenly over everything becomes a skim, and a skim finds the obvious errors and misses the expensive ones.\n\nSo the question is not how to check more. It is how to spend a fixed amount of attention where it matters.\n\n## The rule: cost of being wrong, not confidence of the answer\n\nThe instinct is to check what sounds shaky. That is backwards twice over. Fluency is not correlated with correctness, and fabrication is often *more* fluent than the truth. Meanwhile the answer you did not question is the one that goes into the document.\n\nThe right sorting variable is: **if this is wrong, what happens?**\n\n## Three tiers\n\n**Tier one: no meaningful cost.** A first draft you will rewrite, a brainstorm, an explanation you will test against your own understanding, a summary of something you are about to read anyway. **Check nothing.** Move fast. This is most of what most people use these tools for, and treating it with ceremony is how people conclude the tools are more trouble than they are worth.\n\n**Tier two: recoverable cost.** An internal document, a first pass at analysis, a message to a colleague who will notice an error. **Spot-check.** One number traced to source. One quote searched. One order-of-magnitude check. Two minutes.\n\n**Tier three: expensive or irreversible.** Something a client, a regulator, a court, a patient or the public will see. Money moving. A deadline. A decision about a person. Anything you sign. **Verify every load-bearing claim independently**, against something that is not a language model: the original document, the regulator's page, a spreadsheet, a person who knows.\n\nMost people apply tier two effort to tier three tasks and tier two effort to tier one tasks. Correcting both directions is the whole improvement.\n\n> Figure: Three tiers, sorted by the cost of being wrong. Layers, top to bottom: Tier three: expensive or irreversible (a client, a regulator, a patient, money, a signature. Verify every load-bearing claim against something that is not a model.); Tier two: recoverable (an internal document, a first pass. One number traced, one quote searched, one magnitude check. Two minutes.); Tier one: no meaningful cost (a first draft, a brainstorm, an explanation you will test yourself. Check nothing. Move fast.). Most people apply tier-two effort to everything. Moving the ceremony off tier one and the real verification onto tier three is the whole improvement.\n\n## The two questions to ask before you send the prompt\n\n**1. What would I do differently if this answer were wrong?**\n\nIf the answer is \"nothing much\", you are in tier one and can stop thinking about verification.\n\n**2. Would I be able to tell?**\n\nThis is the one that decides tier three. An answer you could not evaluate even if you read it carefully is qualitatively different from one where an error would be obvious. Combining *expensive if wrong* with *I could not tell* is the situation to refuse, not to check harder, but to get a source or a person who can tell.\n\n## Verification is a design decision, not a habit\n\nEverything in this module is cheaper when it was planned before the prompt was sent.\n\nAsk for quotes, and checking is a text search. Ask for sources beside claims, and checking is opening one link. Ask for the basis of every figure, and a mismatch is visible in a column. Ask for enumeration first, and omissions surface as a count.\n\nDeciding to verify afterwards means doing the work yourself. Deciding beforehand means the output arrives shaped for checking, and the whole thing takes two minutes instead of twenty.\n\n## What does not count as checking\n\n- Reading it again and finding it convincing.\n- Asking the model whether it is sure.\n- Noting that it sounds confident.\n- Noting that it sounds hedged.\n- A second model agreeing, on its own.\n- A green self-check on rules you wrote.\n\nEach of these produces the feeling of verification without the substance, and the feeling is the problem: it consumes the attention that a real check would have needed.\n\n## The sentence to keep\n\n**Decide, before you ask, what you would do if this answer were wrong.**\n\nIf the answer is \"nothing much\", move fast and stop worrying. If it is \"I would send a wrong figure to a client\", \"I would miss a filing deadline\", or \"somebody would take the wrong dose\", then the model's job is to tell you where to look, and something that is not a model has to confirm it.\n\nThat single distinction protects you better than every technique in this course put together.\n\n## And the tier nobody talks about\n\nThere is a fourth category worth naming: things where the answer is fine and the *act of asking* was the problem: the confidential document you pasted, the client name in a prompt, the medical detail now in a memory store.\n\nNo amount of checking the output addresses that, because the risk was incurred on the way in. It belongs in the same decision, made at the same moment: before you send, ask both what happens if the answer is wrong, and what happens if this text is stored, logged, or used to train something. The second question takes two seconds and has no version that can be asked afterwards."},"check":{"question":"Which task most clearly demands the highest tier of verification?","options":[{"text":"A long research summary on an unfamiliar subject, for the user's own background reading.","correct":false,"tempting":"Unfamiliarity is uncomfortable and the consequence is low; errors are corrected as they learn more."},{"text":"A batch of 200 product descriptions for an internal review before publication.","correct":false,"tempting":"A review stage stands between the output and the consequence, which is what tier two is for."},{"text":"A dosage conversion for a medication, where the user could not tell from reading whether it is right.","correct":true},{"text":"A financial model where every figure came from the user's own spreadsheet.","correct":false,"tempting":"Traceable inputs and a checkable structure; important, and the error would be visible on inspection."}]},"prev":{"slug":"checking-what-is-missing","title":"The hardest check: what was left out","url":"https://addaly.com/learn/prompting/checking-what-is-missing"},"next":{"slug":"drafting-in-your-own-words","title":"Drafting without losing the thing that was yours","url":"https://addaly.com/learn/prompting/drafting-in-your-own-words"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'How much checking is enough', in Getting Real Work out of a Model. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/prompting/how-much-checking (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"How much checking is enough","courseTitle":"Getting Real Work out of a Model","module":{"index":7,"slug":"checking-the-answer","title":"Checking the answer"},"answer":null,"keyIdea":"Sort by what happens if the answer is wrong rather than by how confident it sounds, and design the prompt so the checking is a text search rather than a re-reading.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"stack","title":"Three tiers, sorted by the cost of being wrong","layers":[{"label":"Tier three: expensive or irreversible","detail":"a client, a regulator, a patient, money, a signature. Verify every load-bearing claim against something that is not a model."},{"label":"Tier two: recoverable","detail":"an internal document, a first pass. One number traced, one quote searched, one magnitude check. Two minutes."},{"label":"Tier one: no meaningful cost","detail":"a first draft, a brainstorm, an explanation you will test yourself. Check nothing. Move fast."}],"caption":"Most people apply tier-two effort to everything. Moving the ceremony off tier one and the real verification onto tier three is the whole improvement."},"alt":"Figure: Three tiers, sorted by the cost of being wrong. Layers, top to bottom: Tier three: expensive or irreversible (a client, a regulator, a patient, money, a signature. Verify every load-bearing claim against something that is not a model.); Tier two: recoverable (an internal document, a first pass. One number traced, one quote searched, one magnitude check. Two minutes.); Tier one: no meaningful cost (a first draft, a brainstorm, an explanation you will test yourself. Check nothing. Move fast.). Most people apply tier-two effort to everything. Moving the ceremony off tier one and the real verification onto tier three is the whole improvement."}},"blocks":[{"t":"h2","id":"checking-everything-is-the-same-as-checking-nothing","runs":[{"text":"Checking everything is the same as checking nothing"}]},{"t":"p","runs":[{"text":"Nobody verifies every claim in every answer. People who say they do are describing an intention. What actually happens is that effort spread evenly over everything becomes a skim, and a skim finds the obvious errors and misses the expensive ones."}]},{"t":"p","runs":[{"text":"So the question is not how to check more. It is how to spend a fixed amount of attention where it matters."}]},{"t":"h2","id":"the-rule-cost-of-being-wrong-not-confidence-of-the-answer","runs":[{"text":"The rule: cost of being wrong, not confidence of the answer"}]},{"t":"p","runs":[{"text":"The instinct is to check what sounds shaky. That is backwards twice over. Fluency is not correlated with correctness, and fabrication is often "},{"text":"more","i":true},{"text":" fluent than the truth. Meanwhile the answer you did not question is the one that goes into the document."}]},{"t":"p","runs":[{"text":"The right sorting variable is: "},{"text":"if this is wrong, what happens?","b":true}]},{"t":"h2","id":"three-tiers","runs":[{"text":"Three tiers"}]},{"t":"p","runs":[{"text":"Tier one: no meaningful cost.","b":true},{"text":" A first draft you will rewrite, a brainstorm, an explanation you will test against your own understanding, a summary of something you are about to read anyway. "},{"text":"Check nothing.","b":true},{"text":" Move fast. This is most of what most people use these tools for, and treating it with ceremony is how people conclude the tools are more trouble than they are worth."}]},{"t":"p","runs":[{"text":"Tier two: recoverable cost.","b":true},{"text":" An internal document, a first pass at analysis, a message to a colleague who will notice an error. "},{"text":"Spot-check.","b":true},{"text":" One number traced to source. One quote searched. One order-of-magnitude check. Two minutes."}]},{"t":"p","runs":[{"text":"Tier three: expensive or irreversible.","b":true},{"text":" Something a client, a regulator, a court, a patient or the public will see. Money moving. A deadline. A decision about a person. Anything you sign. "},{"text":"Verify every load-bearing claim independently","b":true},{"text":", against something that is not a language model: the original document, the regulator’s page, a spreadsheet, a person who knows."}]},{"t":"p","runs":[{"text":"Most people apply tier two effort to tier three tasks and tier two effort to tier one tasks. Correcting both directions is the whole improvement."}]},{"t":"h2","id":"the-two-questions-to-ask-before-you-send-the-prompt","runs":[{"text":"The two questions to ask before you send the prompt"}]},{"t":"p","runs":[{"text":"1. What would I do differently if this answer were wrong?","b":true}]},{"t":"p","runs":[{"text":"If the answer is “nothing much”, you are in tier one and can stop thinking about verification."}]},{"t":"p","runs":[{"text":"2. Would I be able to tell?","b":true}]},{"t":"p","runs":[{"text":"This is the one that decides tier three. An answer you could not evaluate even if you read it carefully is qualitatively different from one where an error would be obvious. Combining "},{"text":"expensive if wrong","i":true},{"text":" with "},{"text":"I could not tell","i":true},{"text":" is the situation to refuse, not to check harder, but to get a source or a person who can tell."}]},{"t":"h2","id":"verification-is-a-design-decision-not-a-habit","runs":[{"text":"Verification is a design decision, not a habit"}]},{"t":"p","runs":[{"text":"Everything in this module is cheaper when it was planned before the prompt was sent."}]},{"t":"p","runs":[{"text":"Ask for quotes, and checking is a text search. Ask for sources beside claims, and checking is opening one link. Ask for the basis of every figure, and a mismatch is visible in a column. Ask for enumeration first, and omissions surface as a count."}]},{"t":"p","runs":[{"text":"Deciding to verify afterwards means doing the work yourself. Deciding beforehand means the output arrives shaped for checking, and the whole thing takes two minutes instead of twenty."}]},{"t":"h2","id":"what-does-not-count-as-checking","runs":[{"text":"What does not count as checking"}]},{"t":"ul","items":[[{"text":"Reading it again and finding it convincing."}],[{"text":"Asking the model whether it is sure."}],[{"text":"Noting that it sounds confident."}],[{"text":"Noting that it sounds hedged."}],[{"text":"A second model agreeing, on its own."}],[{"text":"A green self-check on rules you wrote."}]]},{"t":"p","runs":[{"text":"Each of these produces the feeling of verification without the substance, and the feeling is the problem: it consumes the attention that a real check would have needed."}]},{"t":"h2","id":"the-sentence-to-keep","runs":[{"text":"The sentence to keep"}]},{"t":"p","runs":[{"text":"Decide, before you ask, what you would do if this answer were wrong.","b":true}]},{"t":"p","runs":[{"text":"If the answer is “nothing much”, move fast and stop worrying. If it is “I would send a wrong figure to a client”, “I would miss a filing deadline”, or “somebody would take the wrong dose”, then the model’s job is to tell you where to look, and something that is not a model has to confirm it."}]},{"t":"p","runs":[{"text":"That single distinction protects you better than every technique in this course put together."}]},{"t":"h2","id":"and-the-tier-nobody-talks-about","runs":[{"text":"And the tier nobody talks about"}]},{"t":"p","runs":[{"text":"There is a fourth category worth naming: things where the answer is fine and the "},{"text":"act of asking","i":true},{"text":" was the problem: the confidential document you pasted, the client name in a prompt, the medical detail now in a memory store."}]},{"t":"p","runs":[{"text":"No amount of checking the output addresses that, because the risk was incurred on the way in. It belongs in the same decision, made at the same moment: before you send, ask both what happens if the answer is wrong, and what happens if this text is stored, logged, or used to train something. The second question takes two seconds and has no version that can be asked afterwards."}]}],"check":{"question":[{"text":"Which task most clearly demands the highest tier of verification?"}],"options":[{"text":[{"text":"A long research summary on an unfamiliar subject, for the user’s own background reading."}],"correct":false,"tempting":[{"text":"Unfamiliarity is uncomfortable and the consequence is low; errors are corrected as they learn more."}]},{"text":[{"text":"A batch of 200 product descriptions for an internal review before publication."}],"correct":false,"tempting":[{"text":"A review stage stands between the output and the consequence, which is what tier two is for."}]},{"text":[{"text":"A dosage conversion for a medication, where the user could not tell from reading whether it is right."}],"correct":true,"tempting":null},{"text":[{"text":"A financial model where every figure came from the user’s own spreadsheet."}],"correct":false,"tempting":[{"text":"Traceable inputs and a checkable structure; important, and the error would be visible on inspection."}]}],"order":[3,2,1,0],"why":null},"prev":{"slug":"checking-what-is-missing","title":"The hardest check: what was left out","minutes":9},"next":{"slug":"drafting-in-your-own-words","title":"Drafting without losing the thing that was yours","minutes":9}}}