Checking a grounded answer in under a minute

The check has to be mechanical

You have grounded the system, the answers carry citations, and the whole thing is working. The remaining question is how you satisfy yourself, quickly, that any particular answer is real.

Reading is not enough. A fabricated answer and a correct one read identically, which has been the point of this course from the beginning. What you need is a procedure short enough that you will actually do it every time.

The sixty-second check

1. Is there a citation at all? No citation means the sentence is unsupported, whatever it says. Some tools cite only some claims; treat an uncited sentence as memory.

2. Does the quoted text exist? Search the source document for a distinctive phrase from the quote. This is the step that catches fabricated citations, and it takes about ten seconds with a find box.

3. Does the quote support the claim? The commonest real-world failure is not an invented quote. It is a genuine quote that says something adjacent. The passage is about deposits; the answer is about final payments. Both are real, and the answer is wrong.

4. Is the source current? Look at the date in the document. A perfect quote from a superseded version is a wrong answer with excellent provenance.

5. Would a missing document change this? Especially for any answer of the form “there is no...”. Absence findings do not survive this step, ever.

Five checks, under a minute for a short answer, and mechanical enough to hand to somebody else.

Making the system easier to check

Ask for the format that makes checking fast, and it costs you nothing:

Answer using only the sources. After each sentence, put
[document name, section] and the exact quoted text it
came from. If a sentence is not supported by a source,
mark it [UNSUPPORTED] and keep it separate at the end.

That last clause is worth more than it looks. Rather than forbidding unsupported statements (which usually just makes them disappear into the supported ones) you give them somewhere to go. Useful context ends up in a clearly-labelled section you can read differently, which is a far better outcome than a confident refusal or a blended answer where you cannot tell the parts apart.

When to check everything, and when to sample

Check every answer when it goes to a customer, when money or a commitment is involved, when the question is unusual, or during the first fortnight with any new setup.

Sample for internal, routine, low-stakes use once the system has a track record. Ten answers a week, checked properly, is enough to notice a change.

The thing to avoid is the middle: a vague intention to check when something looks odd. Nothing looks odd. That is the whole problem.

What a good answer looks like

It is worth knowing what you are aiming at, because it is not the most impressive-looking output.

A good grounded answer is often slightly awkward. It says “the terms document, section 4, states a fourteen-day return window” instead of a smooth paragraph about your excellent returns policy. It says NOT IN SOURCES for the part you did not document. It carries quotes that turn out, when you look, to say exactly what it claimed.

The polished, comprehensive, entirely confident answer with no gaps and no hedging is the one to look at hardest. Real documents have holes in them, and a system that never reports a hole is not reporting.

Teach the check, not the tool

If somebody else will use the system, the five checks are what you hand over, not a demonstration of how good the answers are.

Write them on one card next to the screen. Show a real failure: an answer whose quote turns out to be about the adjacent policy. People generalise from one concrete failure far better than from a warning, and it takes two minutes.

Then agree what happens when a check fails. Usually: do not send it, fix the document if the document was the problem, and tell whoever owns the library. Without that last step, the same failure is caught fifteen separate times by fifteen different people and fixed by none of them.

When the checking never gets cheaper

If, after a month, the five checks still take real effort on every answer, that is information. It means the questions being asked are not ones your documents answer, and the honest response is either to write the missing document or to stop using the system for that class of question.

A grounded assistant should get easier to supervise as your library matures, because the answers converge on material you have written deliberately. If supervision is not getting cheaper, the library is the problem, not your diligence.