Building the failure into the format

A blank is better than a guess

The most expensive output failure is not an error message. It is a well-formed answer containing something the model made up, sitting in a column beside forty things it did not.

You cannot prevent invention with a prohibition, as you know. What you can do is build a legal way to fail into the shape of the output, so that failing is easier than inventing.

Four devices, all cheap.

1. A stated empty value

Every field gets a legal absence.

Where the document does not state a value, write NOT_STATED. Do not infer, estimate, or use a typical value.

The words matter. “Do not infer” blocks the reasonable-sounding deduction. “Do not use a typical value” blocks the most common failure of all, which is a standard figure appearing where a specific one was missing.

Then you can find them: sort by that column and every gap is visible in one glance. A gap you can see is a gap you can go and fill from the original.

2. A source column

For anything extracted from a document, ask where it came from.

Columns: field, value, and the exact quoted sentence it comes from. If you cannot quote a sentence, the value is NOT_STATED.

This is the strongest single technique in this course for document work, and the reason is structural rather than motivational: a quote is checkable in four seconds with a search, and a value that has no quotable source cannot be produced without the absence being visible.

You will also find that requiring a quote reduces invention, because there is nowhere for an unsupported value to sit.

3. A flag rather than a hedge

Hedging inside prose is invisible at scale. “This may be approximately 40,000” reads as 40,000 by the third row.

Give uncertainty a field of its own:

Add a column confidence with one of: stated (explicit in the document), derived (calculated from stated figures; show the calculation), uncertain (interpretation required; say why).

Three values, mechanically defined. That is much more useful than a percentage, which a model will produce and which is not calibrated to anything. You can filter on uncertain and check only those rows.

4. A refusal that is a valid answer

Give “this cannot be done” a shape, so it is not a conversational failure:

If the question cannot be answered from the material provided, reply with exactly: CANNOT ANSWER: followed by what is missing.

Without a permitted shape for failure, the model produces an answer, because producing an answer is what the exchange calls for. With one, “cannot answer” competes on equal terms.

The second half, what is missing, turns a dead end into your next action. Nine times in ten it names a document you have and did not paste.

What this looks like put together

For each of the 12 policies, output:
policy_id | premium | excess | maternity_covered | source_quote | confidence

Rules:
- source_quote: the exact sentence the value came from, or empty.
- If there is no sentence to quote, the value is NOT_STATED.
- confidence: stated / derived / uncertain.
- If a whole policy document is missing or unreadable, output the row
  as CANNOT READ and nothing else for that policy.

Now three failure modes are visible without reading anything: missing values are NOT_STATED, shaky values are uncertain, missing documents are CANNOT READ. What is left is checkable in the time it takes to scan a column.

Why this beats checking harder

You cannot verify forty rows carefully. Nobody does, whatever they say at the start.

What you can do is make the rows that need attention identify themselves, and then check those properly. That is a format decision, made before you send the prompt, and it is the difference between a batch you can trust and a batch you have merely received.

The honest limit

None of this makes the values right. A quoted sentence can be quoted accurately and read wrongly. A row marked stated can point at the wrong stated figure: the subtotal rather than the total, the order date rather than the invoice date.

What these devices buy is visibility, which is a smaller claim and a more useful one. They separate the rows where something is missing or shaky from the rows where something is asserted with a source, and they hand you a five-minute check instead of an unperformable forty-minute one. The remaining errors are the ones that look exactly like correct answers, and those are the subject of a later module.