A schema constrains the shape, not the truth

How the guarantee is produced

When you ask a provider for JSON matching a schema, most of them do not ask politely and hope. They constrain the sampler. At each step the model produces a distribution over the next token, and the runtime masks out every token that would make the partial string impossible to complete into a valid document. If the schema says the next thing must be a key from a fixed set, only those tokens survive the mask.

That is why structured output is reliable in a way prompt instructions are not. It is also exactly why the guarantee is narrower than people assume. Masking operates on the grammar. It knows what shapes are legal. It knows nothing about invoices.

Tool calling is the same machinery wearing a different name: a schema per tool, the model emits an object matching one of them, and your harness dispatches on it.

The failure this creates

Take an extraction schema with invoice_total as a number, currency as a three-letter enum, and due_date as a required string. Run it over real scanned invoices, a quarter of which print no due date at all.

The model cannot omit the field: omission would make the object invalid, and the mask forbids it. It cannot say “not present”, because that is not in the grammar either. So it emits a plausible date. Thirty days after the invoice date is the common guess, because that is what most training-data invoices say, and it will be wrong about a quarter of the time in a field that looks completely normal downstream.

The bug is not the model’s. It is in the schema. Make “I did not find this” representable.

{
  "due_date": { "type": ["string", "null"] },
  "due_date_source": { "enum": ["printed", "inferred_from_terms", "absent"] }
}

Now the honest answer is legal, and you have a field you can filter on. Every extraction schema should have a route for absence, and every downstream step should treat inferred differently from printed.

Enums, flatness, and field order

Three practical rules that repay themselves quickly.

Close the set wherever you can. A category of type string invites forty spellings of the same six categories. An enum of six gives you six, guaranteed, and the failure becomes visible as a wrong choice rather than as silent drift.

Keep it flat. Deeply nested objects with optional branches produce worse extractions than a flat object with prefixed names. This is the consistent report from practitioners rather than a well-published measurement, so treat it as a strong prior to test on your own data, not a law.

Order the fields deliberately. Generation is left to right, and everything already emitted conditions what comes next. A field placed before the answer can act as scratch space; a field placed after cannot influence it. So this works:

{ "evidence_quote": "...", "reasoning": "...", "amount": 4820.00 }

and the same object with amount first does not, because by the time the model writes its reasoning the number is already committed and the reasoning becomes an explanation of whatever was said.

Constrained decoding can cost you quality

If a task needs the model to work something out, forcing it straight into a rigid object removes the room to do that. Two fixes: give it a reasoning field first, as above, or split into two calls: one open call that thinks, one constrained call that formats the result of the first. The second is more tokens and usually more accurate on arithmetic-heavy extraction.

Validate anyway

The schema is a syntax check. Your validator is the semantic one, and it runs in your code after the response arrives:

1. Parse. It will parse; assert it anyway, because streaming, truncation and provider errors all produce partial documents.
2. Types and ranges. A total of 4820000000 in a schema that says number is legal JSON and an obvious data error.
3. Cross-field arithmetic. Line items should sum to the total. Say so in code.
4. Provenance. Every number and identifier should be findable in the source text you sent. If it is not there, the model produced it, and you should know that before it reaches a ledger.

The free path

You do not need a provider’s structured-output feature. Outlines, llama.cpp with a GBNF grammar, and guidance all do constrained generation against local models, and jsonschema or Pydantic in Python validates whatever comes back regardless of who generated it. Pydantic is worth learning on its own: the model class doubles as the schema you send and the validator you run on return, which keeps the two from drifting apart: a drift that produces the worst kind of bug, where the contract you enforce and the contract you asked for quietly differ.