Why it invents things, and when

Fabrication is the system working normally

A model that produces a fake reference is not malfunctioning. It is doing exactly what it was trained to do: produce the most plausible continuation. The most plausible continuation of a citation for the claim that spaced repetition improves retention is is a citation-shaped string. Author surname, initials, a year, a journal that publishes that sort of work, a plausible volume and page range. Every part is drawn from a well-learned pattern. The combination has never existed.

This is the single most useful thing to understand about these tools, because it tells you exactly where the risk lives. The risk is not in the reasoning. It is in the specifics.

The shape of the risk

Rank these by how likely the model is to be wrong, and the ranking is stable across every model you will use:

1. Very safe. General explanations of well-established material. What photosynthesis does. Why a for-loop is slower than vectorised code. The structure of a five-paragraph argument.
2. Usually safe. Standard worked methods. How to integrate by parts. How to balance a redox equation. These are procedures repeated thousands of times in the training text.
3. Risky. Anything with a number, a name, a date, or a page reference attached. Population of a city in a particular year. Which scholar first argued a position. The exact wording of a statute.
4. Very risky. Anything that must exist in the world for your claim to hold: a paper, a URL, a quotation, a court case, a library function, a command-line flag.

Categories 3 and 4 are where the failures live, and they are also the parts a marker checks.

Why it sounds certain

There is no confidence dial connected to the output text. The model produces the tone that fits the context, and academic-sounding questions have academic-sounding answers in the training data, which are written confidently. A hedge like I am not sure, but appears only where hedges appeared in similar text, not where the model’s internal probabilities were low.

Internally, models do carry something like uncertainty: the probability assigned to each token. Research on this has a name, calibration, and models are moderately calibrated on multiple-choice questions and considerably worse on open text. But you cannot see those numbers in a chat window, and the model does not read them back to itself before writing. So confident phrasing carries almost no information about correctness.

Asking are you sure? is not a check. It is another prompt, and the most plausible continuation of a challenge is often an apology and a different answer, whether or not the first one was wrong.

The two questions that do work

The useless move is to ask the model to grade its own answer. Two things do help.

Ask it to produce something checkable. Not is this right but give me the steps, give me the DOI, write code that computes this. A step, a DOI and a program can each be tested against something outside the conversation.

Ask it twice, cold. Open a new chat. Rephrase the question so it does not echo the first answer. Genuine general knowledge is stable across rephrasings; fabrication usually is not. If you get two different citations for the same claim, at least one is invented and probably both are.

What retrieval does and does not fix

Many tools now search the web and attach links. This helps, and it changes the failure rather than removing it. Now the model is summarising retrieved pages, so the summary can misstate a page that is itself correct, and the link can point at a real page that does not contain the claim. Studies of search-grounded assistants keep finding the same pattern: the citations exist, and a meaningful fraction do not support the sentence they are attached to.

The rule survives the upgrade. Open the link. If you did not open it, you did not check it.

Doing this without paying for anything

Everything above works on free tiers. For verification specifically, the free tools that matter are Google Scholar, the DOI resolver at doi.org, arXiv, PubMed, and Unpaywall’s browser extension, which finds legal free copies of paywalled papers. Your library almost certainly has an institutional login too, and the librarian is the most underused expert on your campus.

The habit to build

When you read a model’s answer, mentally underline every proper noun, number and reference. Those are the parts that need a source. The prose around them is usually fine. This takes about ten seconds and catches nearly everything that would otherwise cost you marks.