{"course":{"slug":"context-engineering","title":"Context Engineering","url":"https://addaly.com/learn/context-engineering","level":"advanced","track":"build"},"lesson":{"slug":"citations-that-survive-assembly","title":"Citations you can actually verify","url":"https://addaly.com/learn/context-engineering/citations-that-survive-assembly","position":29,"of":87,"minutes":8,"answer":"Give every passage a short stable handle, keep the mapping outside the window, and verify after generation that each cited handle exists and each quoted span really appears.","keyIdea":"Give every passage a short stable handle, keep the mapping outside the window, and verify after generation that each cited handle exists and each quoted span really appears, because citation presence is easy and citation correctness is not, and only the second is worth showing a user."},"body":{"format":"markdown","text":"## A citation is a claim about your own pipeline\n\nAsking a model to cite its sources is easy. Making the citation mean something requires three things your assembler must provide, and most systems provide only the first.\n\n**A stable handle.** Every passage in the window carries a short marker (`[3]`, `S7`) that the model is told to use. Short matters: it is emitted many times and each character is a token.\n\n**A mapping back.** Your code holds `[3] -> {doc_id, chunk_id, char_span}`. This lives outside the window, in the request's own state, and it is what turns a marker into a link, a page number and a highlight.\n\n**A verification step.** After generation, check the citations before showing them. This is the part that is usually skipped, and it is where the value is.\n\n## Three checks worth running on every answer\n\n```python\ncited = set(re.findall(r\"\\[(\\d+)\\]\", answer))\n\n# 1. Does every cited handle exist?\nbogus = cited - set(handles)          # model cited [12]; you sent 8\n\n# 2. Is every claim-bearing sentence cited?\nuncited = [s for s in sentences(answer) if is_factual(s) and not has_cite(s)]\n\n# 3. Does the quoted text actually appear in the cited passage?\nfor quote, h in quotes_with_handles(answer):\n    assert normalise(quote) in normalise(passage[h].text)\n```\n\nThe first catches a real and common failure: models invent handles, especially when the answer draws on their own background knowledge rather than the passages. A citation to a source you did not send is a strong signal that the sentence is unsupported, and it is detectable with a regex.\n\nThe third is the strongest cheap check available. If the model claims to quote and the string is not there, the sentence is wrong and you can say so without a judge model, a human, or an evaluation set.\n\n## The uncomfortable truth about citation accuracy\n\nA citation says which passage the model was attending to when it wrote the sentence. It does not say the passage supports the sentence.\n\nModels routinely produce sentences that are a blend of a retrieved passage and their own background knowledge, cited to the passage. The citation is not a lie exactly (the passage is on the topic; it did influence the output), but the specific claim may not be in it. This is a well-documented gap: citation *presence* is high, citation *correctness* is meaningfully lower, and the difference does not show up in any metric that counts citations.\n\nSo measure attribution directly, on a sample. Take fifty answered questions, and for each cited sentence ask a human (or a judge model you have checked against a human) whether the cited passage actually supports it. The number that comes back is usually lower than the team expected, and it is the number worth quoting when somebody asks how reliable the assistant is.\n\n## Design so citation is easy for the model\n\nSmall assembly choices change citation quality more than instructions do.\n\n**Number the passages in the window, visibly and consistently.** `[1]` through `[8]`, one marker per passage, the same shape every time. Sequential integers in presentation order beat document ids for one mechanical reason: the model has to copy the marker, and a short token adjacent to the text it labels is far easier to copy correctly than `doc_8831#4`.\n\n**Do not renumber between turns.** In a conversation, if `[3]` means something different on turn four than on turn two, earlier citations in the visible history become wrong. Either keep handles stable per conversation, or resolve them to links immediately and never show raw handles twice.\n\n**Ask for the citation in the same sentence as the claim**, not a bibliography at the end. Trailing lists of sources are almost uncheckable and are where unsupported sentences hide.\n\n**Ask for a short quoted span** where the format allows it. Requiring a quote makes check three possible, and it changes behaviour: a model that must produce a verbatim span is measurably less likely to state something the passage does not contain.\n\n## What to do when verification fails\n\nDecide the policy in advance.\n\nFor a bogus handle or a quote that is not in the passage, the safe options are to regenerate once with the failure pointed out, to strip the sentence, or to show the answer with the unsupported sentence marked. What is not acceptable is showing a citation you know is wrong, because the citation is precisely what makes a user stop checking.\n\nLog every verification failure with the request id and the assembly version. The rate is a quality metric that needs no labels, no judges and no evaluation set: it is computed from artefacts you already have, and it moves when your context assembly changes, which makes it one of the most useful regression signals in the whole system."},"check":{"question":"An assistant cites `[7]` for a specific figure. Passage 7 is on the right topic but contains no such figure. What has most likely happened?","options":[{"text":"The passage was truncated during assembly, so the figure was present in the source but not in the window.","correct":false,"tempting":"Possible, and worth checking, but it would show as a partial passage rather than as a claim the source never made."},{"text":"The handle mapping drifted, so `[7]` in the answer refers to a different passage than the one displayed.","correct":false,"tempting":"A real bug when handles are renumbered between turns, but it would misattribute a claim that some passage does support."},{"text":"The model blended background knowledge with the passage it was attending to and cited the passage.","correct":true},{"text":"The reranker put the wrong passage in position 7, so the model cited the position rather than the content.","correct":false,"tempting":"Confuses ordering with attribution; the model cites the marker attached to the text it read, whatever rank produced it."}]},"prev":{"slug":"multi-hop-and-the-single-chunk-fallacy","title":"Questions no single passage answers","url":"https://addaly.com/learn/context-engineering/multi-hop-and-the-single-chunk-fallacy"},"next":{"slug":"measuring-retrieval-on-its-own","title":"Measuring retrieval without the model","url":"https://addaly.com/learn/context-engineering/measuring-retrieval-on-its-own"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Citations you can actually verify', in Context Engineering. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/context-engineering/citations-that-survive-assembly (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Citations you can actually verify","courseTitle":"Context Engineering","module":{"index":3,"slug":"retrieval-as-a-pipeline","title":"Retrieval as a context pipeline"},"answer":null,"keyIdea":"Give every passage a short stable handle, keep the mapping outside the window, and verify after generation that each cited handle exists and each quoted span really appears, because citation presence is easy and citation correctness is not, and only the second is worth showing a user.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"a-citation-is-a-claim-about-your-own-pipeline","runs":[{"text":"A citation is a claim about your own pipeline"}]},{"t":"p","runs":[{"text":"Asking a model to cite its sources is easy. Making the citation mean something requires three things your assembler must provide, and most systems provide only the first."}]},{"t":"p","runs":[{"text":"A stable handle.","b":true},{"text":" Every passage in the window carries a short marker ("},{"text":"[3]","code":true},{"text":", "},{"text":"S7","code":true},{"text":") that the model is told to use. Short matters: it is emitted many times and each character is a token."}]},{"t":"p","runs":[{"text":"A mapping back.","b":true},{"text":" Your code holds "},{"text":"[3] -> {doc_id, chunk_id, char_span}","code":true},{"text":". This lives outside the window, in the request’s own state, and it is what turns a marker into a link, a page number and a highlight."}]},{"t":"p","runs":[{"text":"A verification step.","b":true},{"text":" After generation, check the citations before showing them. This is the part that is usually skipped, and it is where the value is."}]},{"t":"h2","id":"three-checks-worth-running-on-every-answer","runs":[{"text":"Three checks worth running on every answer"}]},{"t":"code","lang":"python","label":"Python","text":"cited = set(re.findall(r\"\\[(\\d+)\\]\", answer))\n\n# 1. Does every cited handle exist?\nbogus = cited - set(handles)          # model cited [12]; you sent 8\n\n# 2. Is every claim-bearing sentence cited?\nuncited = [s for s in sentences(answer) if is_factual(s) and not has_cite(s)]\n\n# 3. Does the quoted text actually appear in the cited passage?\nfor quote, h in quotes_with_handles(answer):\n    assert normalise(quote) in normalise(passage[h].text)"},{"t":"p","runs":[{"text":"The first catches a real and common failure: models invent handles, especially when the answer draws on their own background knowledge rather than the passages. A citation to a source you did not send is a strong signal that the sentence is unsupported, and it is detectable with a regex."}]},{"t":"p","runs":[{"text":"The third is the strongest cheap check available. If the model claims to quote and the string is not there, the sentence is wrong and you can say so without a judge model, a human, or an evaluation set."}]},{"t":"h2","id":"the-uncomfortable-truth-about-citation-accuracy","runs":[{"text":"The uncomfortable truth about citation accuracy"}]},{"t":"p","runs":[{"text":"A citation says which passage the model was attending to when it wrote the sentence. It does not say the passage supports the sentence."}]},{"t":"p","runs":[{"text":"Models routinely produce sentences that are a blend of a retrieved passage and their own background knowledge, cited to the passage. The citation is not a lie exactly (the passage is on the topic; it did influence the output), but the specific claim may not be in it. This is a well-documented gap: citation "},{"text":"presence","i":true},{"text":" is high, citation "},{"text":"correctness","i":true},{"text":" is meaningfully lower, and the difference does not show up in any metric that counts citations."}]},{"t":"p","runs":[{"text":"So measure attribution directly, on a sample. Take fifty answered questions, and for each cited sentence ask a human (or a judge model you have checked against a human) whether the cited passage actually supports it. The number that comes back is usually lower than the team expected, and it is the number worth quoting when somebody asks how reliable the assistant is."}]},{"t":"h2","id":"design-so-citation-is-easy-for-the-model","runs":[{"text":"Design so citation is easy for the model"}]},{"t":"p","runs":[{"text":"Small assembly choices change citation quality more than instructions do."}]},{"t":"p","runs":[{"text":"Number the passages in the window, visibly and consistently.","b":true},{"text":" "},{"text":"[1]","code":true},{"text":" through "},{"text":"[8]","code":true},{"text":", one marker per passage, the same shape every time. Sequential integers in presentation order beat document ids for one mechanical reason: the model has to copy the marker, and a short token adjacent to the text it labels is far easier to copy correctly than "},{"text":"doc_8831#4","code":true},{"text":"."}]},{"t":"p","runs":[{"text":"Do not renumber between turns.","b":true},{"text":" In a conversation, if "},{"text":"[3]","code":true},{"text":" means something different on turn four than on turn two, earlier citations in the visible history become wrong. Either keep handles stable per conversation, or resolve them to links immediately and never show raw handles twice."}]},{"t":"p","runs":[{"text":"Ask for the citation in the same sentence as the claim","b":true},{"text":", not a bibliography at the end. Trailing lists of sources are almost uncheckable and are where unsupported sentences hide."}]},{"t":"p","runs":[{"text":"Ask for a short quoted span","b":true},{"text":" where the format allows it. Requiring a quote makes check three possible, and it changes behaviour: a model that must produce a verbatim span is measurably less likely to state something the passage does not contain."}]},{"t":"h2","id":"what-to-do-when-verification-fails","runs":[{"text":"What to do when verification fails"}]},{"t":"p","runs":[{"text":"Decide the policy in advance."}]},{"t":"p","runs":[{"text":"For a bogus handle or a quote that is not in the passage, the safe options are to regenerate once with the failure pointed out, to strip the sentence, or to show the answer with the unsupported sentence marked. What is not acceptable is showing a citation you know is wrong, because the citation is precisely what makes a user stop checking."}]},{"t":"p","runs":[{"text":"Log every verification failure with the request id and the assembly version. The rate is a quality metric that needs no labels, no judges and no evaluation set: it is computed from artefacts you already have, and it moves when your context assembly changes, which makes it one of the most useful regression signals in the whole system."}]}],"check":{"question":[{"text":"An assistant cites "},{"text":"[7]","code":true},{"text":" for a specific figure. Passage 7 is on the right topic but contains no such figure. What has most likely happened?"}],"options":[{"text":[{"text":"The passage was truncated during assembly, so the figure was present in the source but not in the window."}],"correct":false,"tempting":[{"text":"Possible, and worth checking, but it would show as a partial passage rather than as a claim the source never made."}]},{"text":[{"text":"The handle mapping drifted, so "},{"text":"[7]","code":true},{"text":" in the answer refers to a different passage than the one displayed."}],"correct":false,"tempting":[{"text":"A real bug when handles are renumbered between turns, but it would misattribute a claim that some passage does support."}]},{"text":[{"text":"The model blended background knowledge with the passage it was attending to and cited the passage."}],"correct":true,"tempting":null},{"text":[{"text":"The reranker put the wrong passage in position 7, so the model cited the position rather than the content."}],"correct":false,"tempting":[{"text":"Confuses ordering with attribution; the model cites the marker attached to the text it read, whatever rank produced it."}]}],"order":[3,2,0,1],"why":null},"prev":{"slug":"multi-hop-and-the-single-chunk-fallacy","title":"Questions no single passage answers","minutes":8},"next":{"slug":"measuring-retrieval-on-its-own","title":"Measuring retrieval without the model","minutes":9}}}