How much checking is enough

Checking everything is the same as checking nothing

Nobody verifies every claim in every answer. People who say they do are describing an intention. What actually happens is that effort spread evenly over everything becomes a skim, and a skim finds the obvious errors and misses the expensive ones.

So the question is not how to check more. It is how to spend a fixed amount of attention where it matters.

The rule: cost of being wrong, not confidence of the answer

The instinct is to check what sounds shaky. That is backwards twice over. Fluency is not correlated with correctness, and fabrication is often more fluent than the truth. Meanwhile the answer you did not question is the one that goes into the document.

The right sorting variable is: if this is wrong, what happens?

Three tiers

Tier one: no meaningful cost. A first draft you will rewrite, a brainstorm, an explanation you will test against your own understanding, a summary of something you are about to read anyway. Check nothing. Move fast. This is most of what most people use these tools for, and treating it with ceremony is how people conclude the tools are more trouble than they are worth.

Tier two: recoverable cost. An internal document, a first pass at analysis, a message to a colleague who will notice an error. Spot-check. One number traced to source. One quote searched. One order-of-magnitude check. Two minutes.

Tier three: expensive or irreversible. Something a client, a regulator, a court, a patient or the public will see. Money moving. A deadline. A decision about a person. Anything you sign. Verify every load-bearing claim independently, against something that is not a language model: the original document, the regulator’s page, a spreadsheet, a person who knows.

Most people apply tier two effort to tier three tasks and tier two effort to tier one tasks. Correcting both directions is the whole improvement.

The two questions to ask before you send the prompt

1. What would I do differently if this answer were wrong?

If the answer is “nothing much”, you are in tier one and can stop thinking about verification.

2. Would I be able to tell?

This is the one that decides tier three. An answer you could not evaluate even if you read it carefully is qualitatively different from one where an error would be obvious. Combining expensive if wrong with I could not tell is the situation to refuse, not to check harder, but to get a source or a person who can tell.

Verification is a design decision, not a habit

Everything in this module is cheaper when it was planned before the prompt was sent.

Ask for quotes, and checking is a text search. Ask for sources beside claims, and checking is opening one link. Ask for the basis of every figure, and a mismatch is visible in a column. Ask for enumeration first, and omissions surface as a count.

Deciding to verify afterwards means doing the work yourself. Deciding beforehand means the output arrives shaped for checking, and the whole thing takes two minutes instead of twenty.

What does not count as checking

• Reading it again and finding it convincing.
• Asking the model whether it is sure.
• Noting that it sounds confident.
• Noting that it sounds hedged.
• A second model agreeing, on its own.
• A green self-check on rules you wrote.

Each of these produces the feeling of verification without the substance, and the feeling is the problem: it consumes the attention that a real check would have needed.

The sentence to keep

Decide, before you ask, what you would do if this answer were wrong.

If the answer is “nothing much”, move fast and stop worrying. If it is “I would send a wrong figure to a client”, “I would miss a filing deadline”, or “somebody would take the wrong dose”, then the model’s job is to tell you where to look, and something that is not a model has to confirm it.

That single distinction protects you better than every technique in this course put together.

And the tier nobody talks about

There is a fourth category worth naming: things where the answer is fine and the act of asking was the problem: the confidential document you pasted, the client name in a prompt, the medical detail now in a memory store.

No amount of checking the output addresses that, because the risk was incurred on the way in. It belongs in the same decision, made at the same moment: before you send, ask both what happens if the answer is wrong, and what happens if this text is stored, logged, or used to train something. The second question takes two seconds and has no version that can be asked afterwards.