Structured output, and what a schema does not buy you

Three levels of forcing

Asking. "Reply with JSON only." Works most of the time, fails on the call where the model opens with "Here's the JSON:" or wraps it in a fence or trails an apology. At 100,000 calls a day, a 0.3% failure rate is 300 broken records.

Tool calling. You declare a function with a JSON Schema; the model emits arguments. Better, because the model was trained on this shape, but most implementations still validate after the fact rather than during generation.

Constrained decoding. At each step the sampler masks every token that cannot appear next in a valid document, so invalid tokens have probability zero. This is what JSON-schema modes, grammar-based sampling (GBNF in llama.cpp), and libraries like Outlines, XGrammar and llguidance actually do. The output *cannot* be malformed. Not "almost never" — cannot.

So the parser problem is solved. That is the whole of what it solves.

The part the schema does not do

Constrained decoding shapes tokens. It knows nothing about the world. {"invoice_total": 0, "currency": "INR"} is perfectly valid and perfectly wrong. Teams turn on strict mode, watch their crash rate go to zero, and report that accuracy "improved" — what improved is that failures stopped being visible as exceptions and started being visible as quiet nonsense in the database.

Worse, a tight constraint can *reduce* accuracy. If your enum is ["refund", "billing", "technical"] and a ticket is about account deletion, the model is forbidden from saying so. It must pick one. You have engineered a guaranteed wrong answer.

Always include an escape member:

{
  "category": {
    "type": "string",
    "enum": ["refund", "billing", "technical", "account", "unknown"]
  }
}

The same goes for every extracted field: null must be reachable, and your prompt must say that "not stated in the document" is a correct answer rather than a failure.

Field order is prompt engineering

Generation is left to right. The model emits your fields in schema order, and each field is conditioned on the ones before it. So this schema:

{ "label": {...}, "reasoning": {"type": "string"} }

produces the label first and *then* a justification for a decision already made. The reasoning is decoration. Reverse it:

{
  "quoted_evidence": {"type": "string"},
  "reasoning":       {"type": "string"},
  "label":           {"type": "string", "enum": [...]},
  "confidence":      {"type": "string", "enum": ["high", "medium", "low"]}
}

and the label is now conditioned on evidence the model had to commit to in writing. This is one of the highest-return changes available in a structured pipeline and it costs you a schema edit. Asking for a quotation first is stronger than asking for reasoning first, because a quotation is checkable — you can assert in code that the string actually appears in the source.

Practical constraints worth knowing

• Providers support different subsets. Recursion, pattern, minimum/maximum, and oneOf are unevenly supported and sometimes silently dropped. Validate after generation regardless of what strict mode promises.
• Depth and unions hurt. A schema with six levels of nesting and a twelve-way union produces worse content than a flat one, even when both are structurally valid. Flatten; run two calls if you must.
• Descriptions are read. Field descriptions land in the prompt. "amount_minor_units": "integer, in the smallest unit — 1500 for ₹15.00, 1500 for $15.00" does real work.
• Streaming is awkward. Partial JSON is not JSON. Use a tolerant incremental parser, or stream a text field and structure the rest at the end.
• Retries need the error. On validation failure, feed the validator message back rather than saying "that was wrong".

for attempt in range(3):
    raw = call(messages, schema=SCHEMA)
    try:
        return Model.model_validate_json(raw)
    except ValidationError as e:
        messages += [
            {"role": "assistant", "content": raw},
            {"role": "user", "content": f"Validation failed:\n{e}\nReturn corrected JSON only."},
        ]
raise ExtractionFailed()

Where it fits

Schemas are how you turn a language model into a component with a type signature. Everything downstream — routing, evaluation, retries, storage, the injection defences in Lesson 8 — gets easier once the output has a shape you can assert on. Just keep the two claims separate in your head: the schema guarantees the shape, and nothing guarantees the truth.