{"course":{"slug":"building-with-ai","title":"Building with AI","url":"https://addaly.com/learn/building-with-ai","level":"intermediate","track":"build"},"lesson":{"slug":"grounding-and-citations","title":"Making the model quote before it answers","url":"https://addaly.com/learn/building-with-ai/grounding-and-citations","position":16,"of":80,"minutes":9,"answer":"Require verbatim quotes before the answer and check in code that each quote appears in the passage it cites, which converts fabrication from invisible to detectable without claiming the answer is therefore true.","keyIdea":"Require verbatim quotes before the answer and check in code that each quote appears in the passage it cites, which converts fabrication from invisible to detectable without claiming the answer is therefore true."},"body":{"format":"markdown","text":"## Why an answer without a quote cannot be checked\n\nGive a model ten passages and a question, and it will produce an answer that blends what it read with what it already knew from training. Both sources produce the same confident prose. Nothing in the output tells you which sentence came from where, and neither will the model if you ask it afterwards: it will generate a plausible attribution the same way it generated the answer.\n\nThe fix is structural rather than persuasive. **Make quoting a required step that happens before the answer, and then verify the quote in code.** That converts an unfalsifiable claim into a testable one.\n\n## The quote-then-answer pattern\n\n```text\nYou are answering strictly from the passages below.\n\nStep 1. Copy out, word for word, every sentence from the passages that\nbears on the question. Put them inside <evidence> tags, each with its\npassage id. Copy exactly; do not paraphrase or correct.\n\nStep 2. Answer using only what is inside <evidence>. If <evidence> is\nempty, output exactly INSUFFICIENT_CONTEXT.\n\n<passages>\n[p1] ...\n[p2] ...\n</passages>\n\nQuestion: {{ q }}\n```\n\nThree properties follow from that ordering.\n\nThe extraction is easy and the model does it well: copying is a much simpler operation than synthesising. The answer is then conditioned on a short, relevant subset instead of ten passages, which measurably improves it. And you now hold a string that either does or does not appear in your source text.\n\n## Verify in code, not in the prompt\n\nThis is the step people skip, and it is the whole point.\n\n```python\nimport re\n\ndef norm(s):\n    return re.sub(r\"\\s+\", \" \", s).strip().lower()\n\ndef check(evidence, passages):\n    bad = [q for q in evidence\n           if norm(q.text) not in norm(passages[q.pid])]\n    return bad          # non-empty means fabricated or altered\n```\n\nNormalise whitespace and case, because models reflow line breaks and fix capitalisation without being asked. Beyond that, be strict. A quote that does not appear verbatim in the passage it names is either invented or silently edited, and both are disqualifying for anything you would put in front of a user as fact.\n\nIn practice, on ordinary retrieval workloads, a small percentage of quotes fail this check, and it is a *different* small percentage than the answers a human reviewer would flag. It catches a failure mode nothing else catches: the confident answer built on a sentence that does not exist.\n\nWhat to do with a failure is a product decision. Retry once, fall back to the sentinel, or show the answer with a warning. Whatever you choose, count it. That count is a real quality metric you can watch move.\n\n## Citations the user can use\n\nOnce you have verified quotes with passage ids, rendering citations is bookkeeping. Two things make the difference between a citation that builds trust and one that erodes it.\n\n**Deep-link to the exact place**, not to the document. `policy.pdf` is not a citation; `policy.pdf#page=14`, with the quoted sentence highlighted, is. A user who clicks and lands on a 60-page PDF has been given the appearance of evidence and none of the substance.\n\n**Cite per claim, not per answer.** A paragraph with one footnote at the end tells the reader nothing about which of its four assertions the source supports. If three sentences come from three passages, mark three.\n\n## The limits, stated honestly\n\nVerified quoting is the strongest cheap defence available, and it is not a truth guarantee. Four gaps you should know about:\n\n- **A correct quote can support a wrong conclusion.** The sentence exists; the inference from it does not follow. Verification catches fabrication, not bad reasoning.\n- **Selective quoting misleads.** Quoting the exception while ignoring the rule above it is technically grounded and substantively false.\n- **Your source can be wrong or stale.** Grounding in a document from 2019 grounds you in 2019.\n- **Nothing here helps if retrieval missed the relevant passage.** The model can only quote what you gave it, and the honest output in that case is the sentinel, which is why the sentinel and the quoting rule belong in the same prompt.\n\nA reasonable reading of those limits: verified quoting takes you from *unknowable* to *auditable*. That is a large step, and it is not the same as *correct*. Say so in your interface. \"Answers are drawn from your documents and every claim links to its source\" is true and useful. \"Answers are accurate\" is a promise no amount of prompt engineering lets you keep."},"check":{"question":"A grounded assistant produces an answer whose quotes all verify against the source passages, yet the answer is wrong. What does this show about the technique?","options":[{"text":"The verification is misconfigured, since correct quotes should imply a correct answer.","correct":false,"tempting":"Assumes an implication that quoting never provided; the check tests provenance of sentences, not validity of inference."},{"text":"Quote verification detects fabricated or altered sources, but a real sentence can be selectively chosen or wrongly reasoned from.","correct":true},{"text":"The model needs a larger thinking budget to reason correctly from the evidence.","correct":false,"tempting":"Proposes a fix for one possible cause while treating the general limitation as an implementation shortfall."},{"text":"The passages should have been summarised before the answer step.","correct":false,"tempting":"Adds a lossy transformation between source and answer, which makes verbatim verification harder rather than making the answer sounder."}]},"prev":{"slug":"prompt-caching","title":"Prompt caching, and the ordering rule that pays for itself","url":"https://addaly.com/learn/building-with-ai/prompt-caching"},"next":{"slug":"steering-refusals-and-tone","title":"Scope, refusals, and a voice that stays put","url":"https://addaly.com/learn/building-with-ai/steering-refusals-and-tone"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Making the model quote before it answers', in Building with AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/building-with-ai/grounding-and-citations (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Making the model quote before it answers","courseTitle":"Building with AI","module":{"index":2,"slug":"prompting-as-engineering","title":"Prompting as engineering"},"answer":null,"keyIdea":"Require verbatim quotes before the answer and check in code that each quote appears in the passage it cites, which converts fabrication from invisible to detectable without claiming the answer is therefore true.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"why-an-answer-without-a-quote-cannot-be-checked","runs":[{"text":"Why an answer without a quote cannot be checked"}]},{"t":"p","runs":[{"text":"Give a model ten passages and a question, and it will produce an answer that blends what it read with what it already knew from training. Both sources produce the same confident prose. Nothing in the output tells you which sentence came from where, and neither will the model if you ask it afterwards: it will generate a plausible attribution the same way it generated the answer."}]},{"t":"p","runs":[{"text":"The fix is structural rather than persuasive. "},{"text":"Make quoting a required step that happens before the answer, and then verify the quote in code.","b":true},{"text":" That converts an unfalsifiable claim into a testable one."}]},{"t":"h2","id":"the-quote-then-answer-pattern","runs":[{"text":"The quote-then-answer pattern"}]},{"t":"code","lang":"text","label":"Text","text":"You are answering strictly from the passages below.\n\nStep 1. Copy out, word for word, every sentence from the passages that\nbears on the question. Put them inside <evidence> tags, each with its\npassage id. Copy exactly; do not paraphrase or correct.\n\nStep 2. Answer using only what is inside <evidence>. If <evidence> is\nempty, output exactly INSUFFICIENT_CONTEXT.\n\n<passages>\n[p1] ...\n[p2] ...\n</passages>\n\nQuestion: {{ q }}"},{"t":"p","runs":[{"text":"Three properties follow from that ordering."}]},{"t":"p","runs":[{"text":"The extraction is easy and the model does it well: copying is a much simpler operation than synthesising. The answer is then conditioned on a short, relevant subset instead of ten passages, which measurably improves it. And you now hold a string that either does or does not appear in your source text."}]},{"t":"h2","id":"verify-in-code-not-in-the-prompt","runs":[{"text":"Verify in code, not in the prompt"}]},{"t":"p","runs":[{"text":"This is the step people skip, and it is the whole point."}]},{"t":"code","lang":"python","label":"Python","text":"import re\n\ndef norm(s):\n    return re.sub(r\"\\s+\", \" \", s).strip().lower()\n\ndef check(evidence, passages):\n    bad = [q for q in evidence\n           if norm(q.text) not in norm(passages[q.pid])]\n    return bad          # non-empty means fabricated or altered"},{"t":"p","runs":[{"text":"Normalise whitespace and case, because models reflow line breaks and fix capitalisation without being asked. Beyond that, be strict. A quote that does not appear verbatim in the passage it names is either invented or silently edited, and both are disqualifying for anything you would put in front of a user as fact."}]},{"t":"p","runs":[{"text":"In practice, on ordinary retrieval workloads, a small percentage of quotes fail this check, and it is a "},{"text":"different","i":true},{"text":" small percentage than the answers a human reviewer would flag. It catches a failure mode nothing else catches: the confident answer built on a sentence that does not exist."}]},{"t":"p","runs":[{"text":"What to do with a failure is a product decision. Retry once, fall back to the sentinel, or show the answer with a warning. Whatever you choose, count it. That count is a real quality metric you can watch move."}]},{"t":"h2","id":"citations-the-user-can-use","runs":[{"text":"Citations the user can use"}]},{"t":"p","runs":[{"text":"Once you have verified quotes with passage ids, rendering citations is bookkeeping. Two things make the difference between a citation that builds trust and one that erodes it."}]},{"t":"p","runs":[{"text":"Deep-link to the exact place","b":true},{"text":", not to the document. "},{"text":"policy.pdf","code":true},{"text":" is not a citation; "},{"text":"policy.pdf#page=14","code":true},{"text":", with the quoted sentence highlighted, is. A user who clicks and lands on a 60-page PDF has been given the appearance of evidence and none of the substance."}]},{"t":"p","runs":[{"text":"Cite per claim, not per answer.","b":true},{"text":" A paragraph with one footnote at the end tells the reader nothing about which of its four assertions the source supports. If three sentences come from three passages, mark three."}]},{"t":"h2","id":"the-limits-stated-honestly","runs":[{"text":"The limits, stated honestly"}],"role":"limit"},{"t":"p","runs":[{"text":"Verified quoting is the strongest cheap defence available, and it is not a truth guarantee. Four gaps you should know about:"}]},{"t":"ul","items":[[{"text":"A correct quote can support a wrong conclusion.","b":true},{"text":" The sentence exists; the inference from it does not follow. Verification catches fabrication, not bad reasoning."}],[{"text":"Selective quoting misleads.","b":true},{"text":" Quoting the exception while ignoring the rule above it is technically grounded and substantively false."}],[{"text":"Your source can be wrong or stale.","b":true},{"text":" Grounding in a document from 2019 grounds you in 2019."}],[{"text":"Nothing here helps if retrieval missed the relevant passage.","b":true},{"text":" The model can only quote what you gave it, and the honest output in that case is the sentinel, which is why the sentinel and the quoting rule belong in the same prompt."}]]},{"t":"p","runs":[{"text":"A reasonable reading of those limits: verified quoting takes you from "},{"text":"unknowable","i":true},{"text":" to "},{"text":"auditable","i":true},{"text":". That is a large step, and it is not the same as "},{"text":"correct","i":true},{"text":". Say so in your interface. “Answers are drawn from your documents and every claim links to its source” is true and useful. “Answers are accurate” is a promise no amount of prompt engineering lets you keep."}]}],"check":{"question":[{"text":"A grounded assistant produces an answer whose quotes all verify against the source passages, yet the answer is wrong. What does this show about the technique?"}],"options":[{"text":[{"text":"The verification is misconfigured, since correct quotes should imply a correct answer."}],"correct":false,"tempting":[{"text":"Assumes an implication that quoting never provided; the check tests provenance of sentences, not validity of inference."}]},{"text":[{"text":"Quote verification detects fabricated or altered sources, but a real sentence can be selectively chosen or wrongly reasoned from."}],"correct":true,"tempting":null},{"text":[{"text":"The model needs a larger thinking budget to reason correctly from the evidence."}],"correct":false,"tempting":[{"text":"Proposes a fix for one possible cause while treating the general limitation as an implementation shortfall."}]},{"text":[{"text":"The passages should have been summarised before the answer step."}],"correct":false,"tempting":[{"text":"Adds a lossy transformation between source and answer, which makes verbatim verification harder rather than making the answer sounder."}]}],"order":[2,1,0,3],"why":null},"prev":{"slug":"prompt-caching","title":"Prompt caching, and the ordering rule that pays for itself","minutes":8},"next":{"slug":"steering-refusals-and-tone","title":"Scope, refusals, and a voice that stays put","minutes":8}}}