{"course":{"slug":"prompting","title":"Getting Real Work out of a Model","url":"https://addaly.com/learn/prompting","level":"beginner","track":"start"},"lesson":{"slug":"citations-and-sources","title":"The citation that does not say that","url":"https://addaly.com/learn/prompting/citations-and-sources","position":56,"of":72,"minutes":9,"answer":"A fabricated citation fits the pattern of a citation better than many real ones, so check that it exists, that it says what is claimed, and that it supports the inference: three separate checks.","keyIdea":"A fabricated citation fits the pattern of a citation better than many real ones, so check that it exists, that it says what is claimed, and that it supports the inference: three separate checks, in that order."},"body":{"format":"markdown","text":"## Three kinds of bad citation, in increasing order of danger\n\n**The invented source.** A paper, a case, a section of an act that does not exist. Authors who never collaborated, a plausible journal, a volume number, a year. This is the famous failure: two New York lawyers were fined in 2023 for filing a brief citing six fabricated cases, and courts in several countries have since sanctioned others. It is also the easiest to catch: search the title, find nothing, done.\n\n**The real source that says something else.** A genuine paper, correctly cited, attached to a claim it does not make, or makes about a different population, at a different dose, in a different decade. Harder, because every checkable feature of the citation passes. You have to read the source.\n\n**The real source, correctly summarised, that does not support the argument.** A study of 40 undergraduates cited as evidence about a national workforce. Nothing is wrong except the inference, and no automated check will ever catch it.\n\nThe first is the one people worry about. The second and third are the ones that get into published work.\n\n> Figure: Three separate checks on a citation, in order. Steps: 1. Does it exist? (search the exact title in quotes; paste the DOI after doi.org and see whether it resolves); 2. Does it say that? (open the abstract and find the claim; check year, population, sample size); 3. Does it support the inference? (forty undergraduates are not a national workforce). The first catches the famous failure and takes five seconds. The second and third catch the ones that reach published work, and no automated check performs them for you.\n\n## Why fabricated references look so convincing\n\nA citation is a highly patterned string: author surnames, a year, a plausible title in the field's register, a journal, a volume, a page range. Producing text that fits a pattern is exactly what the machinery is for, and a fabricated citation is a *better* fit to the pattern than many real ones, because it is assembled from the most typical parts.\n\nThis has a consequence worth holding onto: **fabrication can be more specific and more tidy than the truth**. Detail is not evidence. A vague answer is often more honest than a precise one.\n\n## The thirty-second check\n\nFor each citation you intend to rely on:\n\n1. **Search the exact title in quotation marks.** Nothing found, in Google Scholar or the open web, means it probably does not exist.\n2. **Check the DOI resolves.** Paste `https://doi.org/` plus the DOI. An invented DOI 404s immediately. This takes five seconds and settles it.\n3. **Open the abstract and find the claim.** This is the step people skip, and it is the one that catches the second kind of error.\n4. **Check the details match**: year, population, sample size, whether it is a study or a commentary.\n\nFree tools for all of it: Google Scholar, Semantic Scholar, PubMed for medicine, CourtListener for US case law, and your national legislation website for statutes.\n\n## Prompting so that checking is possible\n\nYou cannot prompt away fabrication. You can make it cheap to detect.\n\n> For every factual claim, give the source. If you are not confident the source exists exactly as cited, write UNVERIFIED beside it. Do not produce a citation you are not confident about: write \"no source\" instead.\n\nThe `UNVERIFIED` flag is not reliable and it is free, and it does surface some of them. What is reliable is the structure: a claim without a source is now visibly a claim without a source.\n\nStronger still, when the material is in front of it:\n\n> Use only the documents I have pasted. Quote the sentence supporting each claim and give the section it came from. If no pasted document supports a claim, do not make the claim.\n\nThat converts an open-ended fabrication risk into a closed-book task where every claim points at something you can find with a text search.\n\n## Search tools change the picture, partly\n\nA model with web search attached fabricates references much less, because it is reporting things it has retrieved. It still misreads them, still attaches a real link to a claim the page does not make, and still cites a page that is itself wrong. Next lesson.\n\n## The rule for anything that goes out\n\nIf a citation appears in something you publish, submit, file or send: you have opened it. Not the model, you.\n\nThis is not a high standard. It is the standard that already applied before any of this existed, and the only thing that has changed is the volume of plausible citations available to somebody in a hurry.\n\n## Where fabrication is likeliest\n\nWorth knowing so you can aim your attention. Invented sources cluster where the training data is thin and the pattern is strong: local case law, non-English scholarship, small journals, statutes from smaller jurisdictions, page numbers, and anything requiring a specific edition. They also cluster where you asked for a specific *number* of sources: \"give me five studies\" pressures the fifth into existence.\n\nAsk for as many sources as exist rather than a target count, and treat any citation from a small or non-English body of literature as unverified until you have opened it."},"check":{"question":"A researcher checks eight citations by confirming each paper exists and each DOI resolves. All eight pass. What class of error remains entirely undetected?","options":[{"text":"Fabricated author names attached to real papers.","correct":false,"tempting":"Would be caught by opening the record, which the described check effectively does."},{"text":"Citations to papers that have since been retracted.","correct":false,"tempting":"A genuine risk and a narrow one, and retraction notices are visible on the record she has already opened."},{"text":"Citations formatted in the wrong style for her journal.","correct":false,"tempting":"A formatting issue, not a class of factual error."},{"text":"A real paper attached to a claim it does not make, or which it makes about a different population or period.","correct":true}]},"prev":{"slug":"spot-a-guess","title":"How to tell when it is guessing","url":"https://addaly.com/learn/prompting/spot-a-guess"},"next":{"slug":"search-grounded-answers","title":"When it can search, and what that fixes","url":"https://addaly.com/learn/prompting/search-grounded-answers"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The citation that does not say that', in Getting Real Work out of a Model. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/prompting/citations-and-sources (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The citation that does not say that","courseTitle":"Getting Real Work out of a Model","module":{"index":7,"slug":"checking-the-answer","title":"Checking the answer"},"answer":null,"keyIdea":"A fabricated citation fits the pattern of a citation better than many real ones, so check that it exists, that it says what is claimed, and that it supports the inference: three separate checks, in that order.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"flow","steps":[{"label":"Does it exist?","detail":"search the exact title in quotes; paste the DOI after doi.org and see whether it resolves"},{"label":"Does it say that?","detail":"open the abstract and find the claim; check year, population, sample size"},{"label":"Does it support the inference?","detail":"forty undergraduates are not a national workforce"}],"title":"Three separate checks on a citation, in order","caption":"The first catches the famous failure and takes five seconds. The second and third catch the ones that reach published work, and no automated check performs them for you."},"alt":"Figure: Three separate checks on a citation, in order. Steps: 1. Does it exist? (search the exact title in quotes; paste the DOI after doi.org and see whether it resolves); 2. Does it say that? (open the abstract and find the claim; check year, population, sample size); 3. Does it support the inference? (forty undergraduates are not a national workforce). The first catches the famous failure and takes five seconds. The second and third catch the ones that reach published work, and no automated check performs them for you."}},"blocks":[{"t":"h2","id":"three-kinds-of-bad-citation-in-increasing-order-of-danger","runs":[{"text":"Three kinds of bad citation, in increasing order of danger"}]},{"t":"p","runs":[{"text":"The invented source.","b":true},{"text":" A paper, a case, a section of an act that does not exist. Authors who never collaborated, a plausible journal, a volume number, a year. This is the famous failure: two New York lawyers were fined in 2023 for filing a brief citing six fabricated cases, and courts in several countries have since sanctioned others. It is also the easiest to catch: search the title, find nothing, done."}]},{"t":"p","runs":[{"text":"The real source that says something else.","b":true},{"text":" A genuine paper, correctly cited, attached to a claim it does not make, or makes about a different population, at a different dose, in a different decade. Harder, because every checkable feature of the citation passes. You have to read the source."}]},{"t":"p","runs":[{"text":"The real source, correctly summarised, that does not support the argument.","b":true},{"text":" A study of 40 undergraduates cited as evidence about a national workforce. Nothing is wrong except the inference, and no automated check will ever catch it."}]},{"t":"p","runs":[{"text":"The first is the one people worry about. The second and third are the ones that get into published work."}]},{"t":"h2","id":"why-fabricated-references-look-so-convincing","runs":[{"text":"Why fabricated references look so convincing"}]},{"t":"p","runs":[{"text":"A citation is a highly patterned string: author surnames, a year, a plausible title in the field’s register, a journal, a volume, a page range. Producing text that fits a pattern is exactly what the machinery is for, and a fabricated citation is a "},{"text":"better","i":true},{"text":" fit to the pattern than many real ones, because it is assembled from the most typical parts."}]},{"t":"p","runs":[{"text":"This has a consequence worth holding onto: "},{"text":"fabrication can be more specific and more tidy than the truth","b":true},{"text":". Detail is not evidence. A vague answer is often more honest than a precise one."}]},{"t":"h2","id":"the-thirty-second-check","runs":[{"text":"The thirty-second check"}]},{"t":"p","runs":[{"text":"For each citation you intend to rely on:"}]},{"t":"ol","start":1,"items":[[{"text":"Search the exact title in quotation marks.","b":true},{"text":" Nothing found, in Google Scholar or the open web, means it probably does not exist."}],[{"text":"Check the DOI resolves.","b":true},{"text":" Paste "},{"text":"https://doi.org/","code":true},{"text":" plus the DOI. An invented DOI 404s immediately. This takes five seconds and settles it."}],[{"text":"Open the abstract and find the claim.","b":true},{"text":" This is the step people skip, and it is the one that catches the second kind of error."}],[{"text":"Check the details match","b":true},{"text":": year, population, sample size, whether it is a study or a commentary."}]]},{"t":"p","runs":[{"text":"Free tools for all of it: Google Scholar, Semantic Scholar, PubMed for medicine, CourtListener for US case law, and your national legislation website for statutes."}]},{"t":"h2","id":"prompting-so-that-checking-is-possible","runs":[{"text":"Prompting so that checking is possible"}]},{"t":"p","runs":[{"text":"You cannot prompt away fabrication. You can make it cheap to detect."}]},{"t":"quote","runs":[{"text":"For every factual claim, give the source. If you are not confident the source exists exactly as cited, write UNVERIFIED beside it. Do not produce a citation you are not confident about: write “no source” instead."}]},{"t":"p","runs":[{"text":"The "},{"text":"UNVERIFIED","code":true},{"text":" flag is not reliable and it is free, and it does surface some of them. What is reliable is the structure: a claim without a source is now visibly a claim without a source."}]},{"t":"p","runs":[{"text":"Stronger still, when the material is in front of it:"}]},{"t":"quote","runs":[{"text":"Use only the documents I have pasted. Quote the sentence supporting each claim and give the section it came from. If no pasted document supports a claim, do not make the claim."}]},{"t":"p","runs":[{"text":"That converts an open-ended fabrication risk into a closed-book task where every claim points at something you can find with a text search."}]},{"t":"h2","id":"search-tools-change-the-picture-partly","runs":[{"text":"Search tools change the picture, partly"}]},{"t":"p","runs":[{"text":"A model with web search attached fabricates references much less, because it is reporting things it has retrieved. It still misreads them, still attaches a real link to a claim the page does not make, and still cites a page that is itself wrong. Next lesson."}]},{"t":"h2","id":"the-rule-for-anything-that-goes-out","runs":[{"text":"The rule for anything that goes out"}]},{"t":"p","runs":[{"text":"If a citation appears in something you publish, submit, file or send: you have opened it. Not the model, you."}]},{"t":"p","runs":[{"text":"This is not a high standard. It is the standard that already applied before any of this existed, and the only thing that has changed is the volume of plausible citations available to somebody in a hurry."}]},{"t":"h2","id":"where-fabrication-is-likeliest","runs":[{"text":"Where fabrication is likeliest"}]},{"t":"p","runs":[{"text":"Worth knowing so you can aim your attention. Invented sources cluster where the training data is thin and the pattern is strong: local case law, non-English scholarship, small journals, statutes from smaller jurisdictions, page numbers, and anything requiring a specific edition. They also cluster where you asked for a specific "},{"text":"number","i":true},{"text":" of sources: “give me five studies” pressures the fifth into existence."}]},{"t":"p","runs":[{"text":"Ask for as many sources as exist rather than a target count, and treat any citation from a small or non-English body of literature as unverified until you have opened it."}]}],"check":{"question":[{"text":"A researcher checks eight citations by confirming each paper exists and each DOI resolves. All eight pass. What class of error remains entirely undetected?"}],"options":[{"text":[{"text":"Fabricated author names attached to real papers."}],"correct":false,"tempting":[{"text":"Would be caught by opening the record, which the described check effectively does."}]},{"text":[{"text":"Citations to papers that have since been retracted."}],"correct":false,"tempting":[{"text":"A genuine risk and a narrow one, and retraction notices are visible on the record she has already opened."}]},{"text":[{"text":"Citations formatted in the wrong style for her journal."}],"correct":false,"tempting":[{"text":"A formatting issue, not a class of factual error."}]},{"text":[{"text":"A real paper attached to a claim it does not make, or which it makes about a different population or period."}],"correct":true,"tempting":null}],"order":[0,1,3,2],"why":null},"prev":{"slug":"spot-a-guess","title":"How to tell when it is guessing","minutes":9},"next":{"slug":"search-grounded-answers","title":"When it can search, and what that fixes","minutes":8}}}