{"course":{"slug":"prompting","title":"Getting Real Work out of a Model","url":"https://addaly.com/learn/prompting","level":"beginner","track":"start"},"lesson":{"slug":"spot-a-guess","title":"How to tell when it is guessing","url":"https://addaly.com/learn/prompting/spot-a-guess","position":55,"of":72,"minutes":9,"answer":"Fabrication can be more specific than truth, so verify by cost of being wrong, never by how confident it sounds.","keyIdea":"Fabrication can be more specific than truth, so verify by cost of being wrong, never by how confident it sounds."},"body":{"format":"markdown","text":"## Confidence is the default setting\n\nA model does not sound unsure when it is unsure. Fluent, calm, well-organised prose is its resting register, and it uses that register for a verified fact and for something it produced whole. You cannot read confidence off the tone, because the tone does not vary with the truth.\n\nSo you need other signals. Here are the ones that actually work.\n\n## Specificity is not evidence\n\nThis is the counterintuitive one, and it is the most useful thing in this lesson.\n\n> Under Section 14(3)(b) of the 2021 Act, filings are due within 21 days.\n\nThat looks like knowledge. Subsection, year, number. In a person, that precision would have been earned by reading the thing.\n\nA fabricated answer can be exactly that specific, because the section number is generated the same way the sentence around it is. Invented citations, invented page numbers, invented case names, invented statistics with a decimal place: these are a well-documented failure mode, and they look more authoritative than the truth, which is often \"it depends on your jurisdiction\".\n\nPrecision tells you nothing about reliability. Treat a suspiciously exact figure as a flag, not a comfort.\n\n## Where guessing concentrates\n\nRisk is not spread evenly. Be most careful with:\n\n- **Anything after the training cutoff.** Prices, office holders, product versions, which company bought whom.\n- **Exact figures.** Fares, tax thresholds, dosages, deadlines, fees. A Delhi metro fare or a Nairobi matatu fare quoted confidently may be four years stale.\n- **Citations and links.** Both the existence and the content.\n- **Local law and local process.** Rules that vary by country, state or city.\n- **Small or local organisations.** A neighbourhood clinic's opening hours were never in the training data in any reliable form.\n- **Anything you are pushing hard for.** If you ask three times for a source and there isn't one, you may get one anyway.\n\n## Four checks that take under a minute\n\n**1. Ask again in a fresh chat.** Same question, new conversation. If you get a different answer, it is guessing. If you get the same answer, that is weak reassurance, not proof: a consistent error is still consistent.\n\n**2. Ask what would make it wrong.** \"Which parts of this are you least sure about, and why?\" and \"What would I need to check before relying on this?\" Models are imperfect at self-assessment but not useless at it, and this often surfaces the shaky sentence. Do not read the answer as a probability.\n\n**3. Check the cheapest verifiable thing.** Open the link. Search the quote. Confirm one number. If a citation turns out to be invented, distrust the whole passage around it, not just that line.\n\n**4. Give it a way out, in advance.** \"If you are not sure, say so.\" \"If the document does not state this, write 'not stated'.\" \"If you need a fact I have not given you, ask.\" Without an alternative to guessing, guessing is what the shape of the exchange calls for.\n\n## The asymmetry worth remembering\n\nA model saying \"I am confident\" tells you almost nothing.\n\nA model saying \"I am not sure about this part\" tells you a little more: it is not a reliable measurement, but volunteering doubt is at least a signal in the right direction, whereas confidence is the baseline.\n\nSo take hedges seriously and take assurances lightly. That is the opposite of how we read people, which is exactly why it takes practice.\n\n## The one habit that matters\n\nDecide, before you ask, what you would do if this answer were wrong.\n\nIf the answer is \"nothing much\" (a first draft, a brainstorm, an explanation you will test against your own understanding) then move fast and do not worry.\n\nIf the answer is \"I would send the wrong figure to a client\", or \"I would miss a filing deadline\", or \"I would take the wrong dose\", then the model's job is to tell you where to look, and something else has to confirm it. Use it to generate the question, not to close it.\n\nThat single distinction will protect you better than any prompt."},"check":{"question":"You ask about a niche regulation in your country. Answer A: \"Under Section 14(3)(b) of the 2021 Act, filings are due within 21 days.\" Answer B: \"Deadlines are usually a few weeks after the event, but I am not certain of the exact rule where you are.\" Which is more likely to be invented, and why?","options":[{"text":"A, because a fabricated detail can be exactly as specific as a real one: the section number is generated the same way the sentence around it is.","correct":true},{"text":"B, because vagueness is what a model produces when it has nothing, so hedging is the real signal that it is making things up.","correct":false,"tempting":"In people, vagueness signals ignorance and precision signals expertise, so we import a rule that works on colleagues and fails badly here."},{"text":"Neither. Wording carries no information about reliability either way, so both are equally likely to be wrong.","correct":false,"tempting":"It sounds like the sophisticated position, but the asymmetry is real: an explicit hedge is weak evidence of genuine uncertainty, while a confident subsection number is a documented failure mode."},{"text":"A, because models are never able to cite legislation correctly.","correct":false,"tempting":"Right answer for the wrong reason: well-known statutes are often cited correctly, so a blanket rule leaves you unable to tell which citations to check."}]},"prev":{"slug":"knowing-when-to-stop","title":"When to stop prompting and do it yourself","url":"https://addaly.com/learn/prompting/knowing-when-to-stop"},"next":{"slug":"citations-and-sources","title":"The citation that does not say that","url":"https://addaly.com/learn/prompting/citations-and-sources"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'How to tell when it is guessing', in Getting Real Work out of a Model. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/prompting/spot-a-guess (Accessed: 31 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"How to tell when it is guessing","courseTitle":"Getting Real Work out of a Model","module":{"index":7,"slug":"checking-the-answer","title":"Checking the answer"},"answer":null,"keyIdea":"Fabrication can be more specific than truth, so verify by cost of being wrong, never by how confident it sounds.","hero":{"kind":"plate","image":"https://addaly.com/art/modules/prompting/checking-the-answer.webp","alt":"A clinic reception desk with a rubber stamp, a ruled sheet, spectacles, an open book and a telephone.","caption":"Verification is a desk habit rather than a feeling: the claim, the register it is checked against, and the stamp that only goes on afterwards."},"blocks":[{"t":"h2","id":"confidence-is-the-default-setting","runs":[{"text":"Confidence is the default setting"}]},{"t":"p","runs":[{"text":"A model does not sound unsure when it is unsure. Fluent, calm, well-organised prose is its resting register, and it uses that register for a verified fact and for something it produced whole. You cannot read confidence off the tone, because the tone does not vary with the truth."}]},{"t":"p","runs":[{"text":"So you need other signals. Here are the ones that actually work."}]},{"t":"h2","id":"specificity-is-not-evidence","runs":[{"text":"Specificity is not evidence"}]},{"t":"p","runs":[{"text":"This is the counterintuitive one, and it is the most useful thing in this lesson."}]},{"t":"quote","runs":[{"text":"Under Section 14(3)(b) of the 2021 Act, filings are due within 21 days."}]},{"t":"p","runs":[{"text":"That looks like knowledge. Subsection, year, number. In a person, that precision would have been earned by reading the thing."}]},{"t":"p","runs":[{"text":"A fabricated answer can be exactly that specific, because the section number is generated the same way the sentence around it is. Invented citations, invented page numbers, invented case names, invented statistics with a decimal place: these are a well-documented failure mode, and they look more authoritative than the truth, which is often “it depends on your jurisdiction”."}]},{"t":"p","runs":[{"text":"Precision tells you nothing about reliability. Treat a suspiciously exact figure as a flag, not a comfort."}]},{"t":"h2","id":"where-guessing-concentrates","runs":[{"text":"Where guessing concentrates"}]},{"t":"p","runs":[{"text":"Risk is not spread evenly. Be most careful with:"}]},{"t":"ul","items":[[{"text":"Anything after the training cutoff.","b":true},{"text":" Prices, office holders, product versions, which company bought whom."}],[{"text":"Exact figures.","b":true},{"text":" Fares, tax thresholds, dosages, deadlines, fees. A Delhi metro fare or a Nairobi matatu fare quoted confidently may be four years stale."}],[{"text":"Citations and links.","b":true},{"text":" Both the existence and the content."}],[{"text":"Local law and local process.","b":true},{"text":" Rules that vary by country, state or city."}],[{"text":"Small or local organisations.","b":true},{"text":" A neighbourhood clinic’s opening hours were never in the training data in any reliable form."}],[{"text":"Anything you are pushing hard for.","b":true},{"text":" If you ask three times for a source and there isn’t one, you may get one anyway."}]]},{"t":"h2","id":"four-checks-that-take-under-a-minute","runs":[{"text":"Four checks that take under a minute"}]},{"t":"p","runs":[{"text":"1. Ask again in a fresh chat.","b":true},{"text":" Same question, new conversation. If you get a different answer, it is guessing. If you get the same answer, that is weak reassurance, not proof: a consistent error is still consistent."}]},{"t":"p","runs":[{"text":"2. Ask what would make it wrong.","b":true},{"text":" “Which parts of this are you least sure about, and why?” and “What would I need to check before relying on this?” Models are imperfect at self-assessment but not useless at it, and this often surfaces the shaky sentence. Do not read the answer as a probability."}]},{"t":"p","runs":[{"text":"3. Check the cheapest verifiable thing.","b":true},{"text":" Open the link. Search the quote. Confirm one number. If a citation turns out to be invented, distrust the whole passage around it, not just that line."}]},{"t":"p","runs":[{"text":"4. Give it a way out, in advance.","b":true},{"text":" “If you are not sure, say so.” “If the document does not state this, write ‘not stated’.” “If you need a fact I have not given you, ask.” Without an alternative to guessing, guessing is what the shape of the exchange calls for."}]},{"t":"h2","id":"the-asymmetry-worth-remembering","runs":[{"text":"The asymmetry worth remembering"}]},{"t":"p","runs":[{"text":"A model saying “I am confident” tells you almost nothing."}]},{"t":"p","runs":[{"text":"A model saying “I am not sure about this part” tells you a little more: it is not a reliable measurement, but volunteering doubt is at least a signal in the right direction, whereas confidence is the baseline."}]},{"t":"p","runs":[{"text":"So take hedges seriously and take assurances lightly. That is the opposite of how we read people, which is exactly why it takes practice."}]},{"t":"h2","id":"the-one-habit-that-matters","runs":[{"text":"The one habit that matters"}]},{"t":"p","runs":[{"text":"Decide, before you ask, what you would do if this answer were wrong."}]},{"t":"p","runs":[{"text":"If the answer is “nothing much” (a first draft, a brainstorm, an explanation you will test against your own understanding) then move fast and do not worry."}]},{"t":"p","runs":[{"text":"If the answer is “I would send the wrong figure to a client”, or “I would miss a filing deadline”, or “I would take the wrong dose”, then the model’s job is to tell you where to look, and something else has to confirm it. Use it to generate the question, not to close it."}]},{"t":"p","runs":[{"text":"That single distinction will protect you better than any prompt."}]}],"check":{"question":[{"text":"You ask about a niche regulation in your country. Answer A: “Under Section 14(3)(b) of the 2021 Act, filings are due within 21 days.” Answer B: “Deadlines are usually a few weeks after the event, but I am not certain of the exact rule where you are.” Which is more likely to be invented, and why?"}],"options":[{"text":[{"text":"A, because a fabricated detail can be exactly as specific as a real one: the section number is generated the same way the sentence around it is."}],"correct":true,"tempting":null},{"text":[{"text":"B, because vagueness is what a model produces when it has nothing, so hedging is the real signal that it is making things up."}],"correct":false,"tempting":[{"text":"In people, vagueness signals ignorance and precision signals expertise, so we import a rule that works on colleagues and fails badly here."}]},{"text":[{"text":"Neither. Wording carries no information about reliability either way, so both are equally likely to be wrong."}],"correct":false,"tempting":[{"text":"It sounds like the sophisticated position, but the asymmetry is real: an explicit hedge is weak evidence of genuine uncertainty, while a confident subsection number is a documented failure mode."}]},{"text":[{"text":"A, because models are never able to cite legislation correctly."}],"correct":false,"tempting":[{"text":"Right answer for the wrong reason: well-known statutes are often cited correctly, so a blanket rule leaves you unable to tell which citations to check."}]}],"order":[3,2,1,0],"why":null},"prev":{"slug":"knowing-when-to-stop","title":"When to stop prompting and do it yourself","minutes":8},"next":{"slug":"citations-and-sources","title":"The citation that does not say that","minutes":9}}}