Turning a document into a row you can defend

Two kinds of error, and only one of them hurts

An extractor makes two mistakes. It misses a field that was there, or it fills a field with something that was not. The first is annoying and visible: a blank appears, somebody looks at the document. The second is a wrong number entering a ledger with no signal attached, and it can survive for a year.

The whole design goal is therefore to convert wrong answers into missing ones. Everything below serves that.

The quote check

Require the model to return, for every field, a verbatim snippet of the source text containing the value:

{
  "invoice_total": "48,200.00",
  "invoice_total_quote": "Total payable  48,200.00",
  "invoice_total_page": 2
}

Then, in code, search the extracted text of page 2 for that quote. Not with a model: with a string search, after collapsing whitespace. If the quote is not there, the field is rejected and set to null with a reason.

This is the strongest cheap check in document extraction, and its power comes from where it sits. Fluency cannot get past it. A model that invents a total must also invent a line of source text, and the invented line will not be present in the document. In practice a large share of hallucinated fields fail this test immediately, and the ones that pass are usually genuine values that were misread rather than imagined: a different and more tractable problem.

Allow for OCR noise by comparing loosely: strip spaces and punctuation, and accept a near match above a high similarity threshold. Do not loosen it further than that, or you have reinvented the problem.

Make absence expressible

As covered in the schema lesson: if a field is required and the value is not on the page, the model will produce something. So every field gets a status (printed, inferred, or absent) and the pipeline treats them differently. Inferred values are usable for triage and never for payment.

Extract as printed, normalise afterwards

Pull "1,20,500.00" and "31 Mar 26" exactly as they appear, then convert in your own code, in a separate step, with the raw string retained beside the parsed value.

Doing both at once looks tidier and destroys your ability to debug. When a total comes out a hundred times too large, you need to know whether the model misread the page or your parser mishandled a separator, and if the printed string was never stored you cannot tell.

Measure per field, never in aggregate

Label fifty to a hundred real documents by hand, an afternoon, and the highest-value afternoon in the project, and score each field separately.

You will find something like: supplier name 99 per cent, invoice number 97, total 96, due date 71, line items 62. A single “94 per cent accurate” headline hides that, and the due date is the field that decides when money moves. Per-field numbers also tell you where to spend: the fix for line items is rarely the fix for dates.

Keep those documents as a fixed set and re-run it whenever you change the prompt, the model or the parser. This is the regression test for the part of the system that has no unit tests.

Design the human step, not just the model step

Nobody automates document extraction to 100 per cent. The realistic target is to route the uncertain fraction to a person efficiently.

Show them the page image with the quote highlighted and the extracted value in an editable box beside it. That is a five to twenty second decision. Sending them a PDF and a spreadsheet and asking them to check is a four-minute decision, and the difference between those two numbers is the difference between a project that saves time and one that moves it.

Send to review: anything with a failed quote check, anything marked inferred, anything where the arithmetic does not reconcile, and a random one in twenty of everything else. The random sample is not optional: it is the only way to learn the error rate among cases the system was confident about.

Multi-page and amended values

A field can appear more than once with different values: a total on page one and a revised total on page four, or an amendment stapled to the front. Decide the rule explicitly (usually the latest occurrence wins, or the one on the page carrying the word “revised”) record which occurrence was used, and flag the disagreement rather than silently choosing. A silent choice between two totals is exactly the kind of decision nobody knows was made until an auditor asks.