There is no truth step
Walk back through the whole pipeline. Text becomes tokens. Tokens become vectors. Blocks add contributions. The final vector produces a score for every possible next token. One is sampled.
Nowhere in that sequence is there a check against the world. There is no database lookup, no verification, no flag that fires when the model reaches beyond what it saw. The machinery that produces a correct fact and the machinery that produces a fabricated one are the same machinery, running identically. From the inside, nothing distinguishes them.
This is why "hallucination" is a slightly misleading word. It suggests a malfunction. What actually happens is the system doing precisely what it does, on an input where the plausible continuation happens not to be true.
Why the invented details look so good
A fabricated citation arrives with a plausible journal, a volume number, a year, a page range and authors who really work in that field. That is not the model trying to deceive you. It is the model producing the shape of a citation, because shape is exactly what it learned. A real citation and a fake one are equally well-formed, so well-formedness cannot separate them, and well-formedness is what the objective optimised.
The same explains why hallucination clusters where it does. Common facts appear thousands of times in training and are strongly represented. Facts that appear twice — the exact filing date of one court case, a specific figure in one company's report, the API signature of a small library — leave a weak trace, and something plausible fills the gap.
Saying "I don't know" is just another token sequence
There is no abstain button. "I am not sure" is a string of tokens that must out-score the fluent answer. Whether it does is decided by the same distribution as everything else.
And training pushes against it. One influential argument holds that the way models are evaluated is much of the cause: benchmarks score an answer as right or wrong, a blank gets zero, and a guess sometimes gets a point. Under that scoring, a model that guesses beats an identical model that abstains, so guessing is what gets selected for, stage after stage. Preference training pulls the same way, since raters tend to prefer confident answers to hedged ones. This account is argued rather than proven, but the incentive it describes is straightforwardly real.
There is a related and better-established finding: models often carry an internal signal that correlates with whether they are about to be wrong. The information exists somewhere in the network. It is just not what governs the output.
What actually helps, honestly rated
Retrieval. Put the source text in the prompt and ask for an answer from it. This is the largest single improvement available, and it is not a cure — models still drift beyond the provided passage, and retrieving the wrong passage produces a confident answer about the wrong thing.
Verified citations. Asking for citations without giving the model a search tool asks for well-shaped citations, which is what you will get. Citations are only worth trusting when something actually fetched the document.
Sampling several times. Ask the same question five times at moderate temperature. Agreement is weak evidence of solid ground; contradiction is strong evidence you are in thin material. Crude, cheap, effective.
A separate checking pass. Giving the answer and the source back to a model and asking whether each claim is supported catches a real fraction of errors, because judging support is an easier task than generating from nothing.
What does not help: telling the model not to hallucinate. It has no lever for that.
The design consequence
Rates fall with better models and better grounding, and they do not reach zero, because nothing in the method contains a step that could make them zero. So build accordingly. In any workflow where a false confident answer is expensive — medical, legal, financial, safety — the model drafts and a human or a deterministic system verifies. That is not a temporary precaution for the current generation of models. It follows from how the thing works.
Before you move on