The citation that does not say that

Three kinds of bad citation, in increasing order of danger

The invented source. A paper, a case, a section of an act that does not exist. Authors who never collaborated, a plausible journal, a volume number, a year. This is the famous failure: two New York lawyers were fined in 2023 for filing a brief citing six fabricated cases, and courts in several countries have since sanctioned others. It is also the easiest to catch: search the title, find nothing, done.

The real source that says something else. A genuine paper, correctly cited, attached to a claim it does not make, or makes about a different population, at a different dose, in a different decade. Harder, because every checkable feature of the citation passes. You have to read the source.

The real source, correctly summarised, that does not support the argument. A study of 40 undergraduates cited as evidence about a national workforce. Nothing is wrong except the inference, and no automated check will ever catch it.

The first is the one people worry about. The second and third are the ones that get into published work.

Why fabricated references look so convincing

A citation is a highly patterned string: author surnames, a year, a plausible title in the field’s register, a journal, a volume, a page range. Producing text that fits a pattern is exactly what the machinery is for, and a fabricated citation is a better fit to the pattern than many real ones, because it is assembled from the most typical parts.

This has a consequence worth holding onto: fabrication can be more specific and more tidy than the truth. Detail is not evidence. A vague answer is often more honest than a precise one.

The thirty-second check

For each citation you intend to rely on:

1. Search the exact title in quotation marks. Nothing found, in Google Scholar or the open web, means it probably does not exist.
2. Check the DOI resolves. Paste https://doi.org/ plus the DOI. An invented DOI 404s immediately. This takes five seconds and settles it.
3. Open the abstract and find the claim. This is the step people skip, and it is the one that catches the second kind of error.
4. Check the details match: year, population, sample size, whether it is a study or a commentary.

Free tools for all of it: Google Scholar, Semantic Scholar, PubMed for medicine, CourtListener for US case law, and your national legislation website for statutes.

Prompting so that checking is possible

You cannot prompt away fabrication. You can make it cheap to detect.

For every factual claim, give the source. If you are not confident the source exists exactly as cited, write UNVERIFIED beside it. Do not produce a citation you are not confident about: write “no source” instead.

The UNVERIFIED flag is not reliable and it is free, and it does surface some of them. What is reliable is the structure: a claim without a source is now visibly a claim without a source.

Stronger still, when the material is in front of it:

Use only the documents I have pasted. Quote the sentence supporting each claim and give the section it came from. If no pasted document supports a claim, do not make the claim.

That converts an open-ended fabrication risk into a closed-book task where every claim points at something you can find with a text search.

Search tools change the picture, partly

A model with web search attached fabricates references much less, because it is reporting things it has retrieved. It still misreads them, still attaches a real link to a claim the page does not make, and still cites a page that is itself wrong. Next lesson.

The rule for anything that goes out

If a citation appears in something you publish, submit, file or send: you have opened it. Not the model, you.

This is not a high standard. It is the standard that already applied before any of this existed, and the only thing that has changed is the volume of plausible citations available to somebody in a hurry.

Where fabrication is likeliest

Worth knowing so you can aim your attention. Invented sources cluster where the training data is thin and the pattern is strong: local case law, non-English scholarship, small journals, statutes from smaller jurisdictions, page numbers, and anything requiring a specific edition. They also cluster where you asked for a specific number of sources: “give me five studies” pressures the fifth into existence.

Ask for as many sources as exist rather than a target count, and treat any citation from a small or non-English body of literature as unverified until you have opened it.