{"course":{"slug":"agents-and-automation","title":"Agents and Automation","url":"https://addaly.com/learn/agents-and-automation","level":"intermediate","track":"build"},"lesson":{"slug":"structured-output-and-schemas","title":"A schema constrains the shape, not the truth","url":"https://addaly.com/learn/agents-and-automation/structured-output-and-schemas","position":11,"of":72,"minutes":9,"answer":"Constrained decoding guarantees the output parses, and nothing else: a required field the model has no evidence for will be invented, because the sampler is not allowed to leave it out.","keyIdea":"Constrained decoding guarantees the output parses, and nothing else: a required field the model has no evidence for will be invented, because the sampler is not allowed to leave it out."},"body":{"format":"markdown","text":"## How the guarantee is produced\n\nWhen you ask a provider for JSON matching a schema, most of them do not ask politely and hope. They constrain the sampler. At each step the model produces a distribution over the next token, and the runtime masks out every token that would make the partial string impossible to complete into a valid document. If the schema says the next thing must be a key from a fixed set, only those tokens survive the mask.\n\nThat is why structured output is reliable in a way prompt instructions are not. It is also exactly why the guarantee is narrower than people assume. Masking operates on the grammar. It knows what shapes are legal. It knows nothing about invoices.\n\nTool calling is the same machinery wearing a different name: a schema per tool, the model emits an object matching one of them, and your harness dispatches on it.\n\n## The failure this creates\n\nTake an extraction schema with `invoice_total` as a number, `currency` as a three-letter enum, and `due_date` as a required string. Run it over real scanned invoices, a quarter of which print no due date at all.\n\nThe model cannot omit the field: omission would make the object invalid, and the mask forbids it. It cannot say \"not present\", because that is not in the grammar either. So it emits a plausible date. Thirty days after the invoice date is the common guess, because that is what most training-data invoices say, and it will be wrong about a quarter of the time in a field that looks completely normal downstream.\n\nThe bug is not the model's. It is in the schema. **Make \"I did not find this\" representable.**\n\n```json\n{\n  \"due_date\": { \"type\": [\"string\", \"null\"] },\n  \"due_date_source\": { \"enum\": [\"printed\", \"inferred_from_terms\", \"absent\"] }\n}\n```\n\nNow the honest answer is legal, and you have a field you can filter on. Every extraction schema should have a route for absence, and every downstream step should treat `inferred` differently from `printed`.\n\n## Enums, flatness, and field order\n\nThree practical rules that repay themselves quickly.\n\n**Close the set wherever you can.** A `category` of type string invites forty spellings of the same six categories. An enum of six gives you six, guaranteed, and the failure becomes visible as a wrong choice rather than as silent drift.\n\n**Keep it flat.** Deeply nested objects with optional branches produce worse extractions than a flat object with prefixed names. This is the consistent report from practitioners rather than a well-published measurement, so treat it as a strong prior to test on your own data, not a law.\n\n**Order the fields deliberately.** Generation is left to right, and everything already emitted conditions what comes next. A field placed *before* the answer can act as scratch space; a field placed after cannot influence it. So this works:\n\n```json\n{ \"evidence_quote\": \"...\", \"reasoning\": \"...\", \"amount\": 4820.00 }\n```\n\nand the same object with `amount` first does not, because by the time the model writes its reasoning the number is already committed and the reasoning becomes an explanation of whatever was said.\n\n## Constrained decoding can cost you quality\n\nIf a task needs the model to work something out, forcing it straight into a rigid object removes the room to do that. Two fixes: give it a reasoning field first, as above, or split into two calls: one open call that thinks, one constrained call that formats the result of the first. The second is more tokens and usually more accurate on arithmetic-heavy extraction.\n\n## Validate anyway\n\nThe schema is a syntax check. Your validator is the semantic one, and it runs in your code after the response arrives:\n\n1. **Parse.** It will parse; assert it anyway, because streaming, truncation and provider errors all produce partial documents.\n2. **Types and ranges.** A total of 4820000000 in a schema that says number is legal JSON and an obvious data error.\n3. **Cross-field arithmetic.** Line items should sum to the total. Say so in code.\n4. **Provenance.** Every number and identifier should be findable in the source text you sent. If it is not there, the model produced it, and you should know that before it reaches a ledger.\n\n## The free path\n\nYou do not need a provider's structured-output feature. **Outlines**, **llama.cpp** with a GBNF grammar, and **guidance** all do constrained generation against local models, and `jsonschema` or **Pydantic** in Python validates whatever comes back regardless of who generated it. Pydantic is worth learning on its own: the model class doubles as the schema you send and the validator you run on return, which keeps the two from drifting apart: a drift that produces the worst kind of bug, where the contract you enforce and the contract you asked for quietly differ."},"check":{"question":"An extraction schema marks `due_date` as a required string, constrained decoding is on, and a quarter of the invoices have no due date printed anywhere. What happens on those invoices, and why?","options":[{"text":"The model emits a plausible date, because the sampler is masked to tokens that keep the object valid and a missing required field is not valid.","correct":true},{"text":"The response fails schema validation and the harness can retry or fall back.","correct":false,"tempting":"Assumes the constraint is checked after generation; it is applied during generation, so an invalid object is never produced to be caught."},{"text":"The model returns an empty string, which is the conventional signal for a missing value.","correct":false,"tempting":"An empty string is legal under the schema, but nothing pushes the model towards it: training data is full of complete invoices, so it fills."},{"text":"The field is left out, since the model has no evidence to support any value.","correct":false,"tempting":"Omission is precisely what \"required\" forbids, and the mask enforces the requirement token by token."}]},"prev":{"slug":"tools-and-function-calling","title":"A tool call is a request, not an action","url":"https://addaly.com/learn/agents-and-automation/tools-and-function-calling"},"next":{"slug":"the-message-list","title":"What the model actually sees each turn","url":"https://addaly.com/learn/agents-and-automation/the-message-list"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'A schema constrains the shape, not the truth', in Agents and Automation. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/agents-and-automation/structured-output-and-schemas (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"A schema constrains the shape, not the truth","courseTitle":"Agents and Automation","module":{"index":2,"slug":"how-the-loop-runs","title":"The machinery: transcript, tools, loop"},"answer":null,"keyIdea":"Constrained decoding guarantees the output parses, and nothing else: a required field the model has no evidence for will be invented, because the sampler is not allowed to leave it out.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"how-the-guarantee-is-produced","runs":[{"text":"How the guarantee is produced"}]},{"t":"p","runs":[{"text":"When you ask a provider for JSON matching a schema, most of them do not ask politely and hope. They constrain the sampler. At each step the model produces a distribution over the next token, and the runtime masks out every token that would make the partial string impossible to complete into a valid document. If the schema says the next thing must be a key from a fixed set, only those tokens survive the mask."}]},{"t":"p","runs":[{"text":"That is why structured output is reliable in a way prompt instructions are not. It is also exactly why the guarantee is narrower than people assume. Masking operates on the grammar. It knows what shapes are legal. It knows nothing about invoices."}]},{"t":"p","runs":[{"text":"Tool calling is the same machinery wearing a different name: a schema per tool, the model emits an object matching one of them, and your harness dispatches on it."}]},{"t":"h2","id":"the-failure-this-creates","runs":[{"text":"The failure this creates"}]},{"t":"p","runs":[{"text":"Take an extraction schema with "},{"text":"invoice_total","code":true},{"text":" as a number, "},{"text":"currency","code":true},{"text":" as a three-letter enum, and "},{"text":"due_date","code":true},{"text":" as a required string. Run it over real scanned invoices, a quarter of which print no due date at all."}]},{"t":"p","runs":[{"text":"The model cannot omit the field: omission would make the object invalid, and the mask forbids it. It cannot say “not present”, because that is not in the grammar either. So it emits a plausible date. Thirty days after the invoice date is the common guess, because that is what most training-data invoices say, and it will be wrong about a quarter of the time in a field that looks completely normal downstream."}]},{"t":"p","runs":[{"text":"The bug is not the model’s. It is in the schema. "},{"text":"Make “I did not find this” representable.","b":true}]},{"t":"code","lang":"json","label":"JSON","text":"{\n  \"due_date\": { \"type\": [\"string\", \"null\"] },\n  \"due_date_source\": { \"enum\": [\"printed\", \"inferred_from_terms\", \"absent\"] }\n}"},{"t":"p","runs":[{"text":"Now the honest answer is legal, and you have a field you can filter on. Every extraction schema should have a route for absence, and every downstream step should treat "},{"text":"inferred","code":true},{"text":" differently from "},{"text":"printed","code":true},{"text":"."}]},{"t":"h2","id":"enums-flatness-and-field-order","runs":[{"text":"Enums, flatness, and field order"}]},{"t":"p","runs":[{"text":"Three practical rules that repay themselves quickly."}]},{"t":"p","runs":[{"text":"Close the set wherever you can.","b":true},{"text":" A "},{"text":"category","code":true},{"text":" of type string invites forty spellings of the same six categories. An enum of six gives you six, guaranteed, and the failure becomes visible as a wrong choice rather than as silent drift."}]},{"t":"p","runs":[{"text":"Keep it flat.","b":true},{"text":" Deeply nested objects with optional branches produce worse extractions than a flat object with prefixed names. This is the consistent report from practitioners rather than a well-published measurement, so treat it as a strong prior to test on your own data, not a law."}]},{"t":"p","runs":[{"text":"Order the fields deliberately.","b":true},{"text":" Generation is left to right, and everything already emitted conditions what comes next. A field placed "},{"text":"before","i":true},{"text":" the answer can act as scratch space; a field placed after cannot influence it. So this works:"}]},{"t":"code","lang":"json","label":"JSON","text":"{ \"evidence_quote\": \"...\", \"reasoning\": \"...\", \"amount\": 4820.00 }"},{"t":"p","runs":[{"text":"and the same object with "},{"text":"amount","code":true},{"text":" first does not, because by the time the model writes its reasoning the number is already committed and the reasoning becomes an explanation of whatever was said."}]},{"t":"h2","id":"constrained-decoding-can-cost-you-quality","runs":[{"text":"Constrained decoding can cost you quality"}]},{"t":"p","runs":[{"text":"If a task needs the model to work something out, forcing it straight into a rigid object removes the room to do that. Two fixes: give it a reasoning field first, as above, or split into two calls: one open call that thinks, one constrained call that formats the result of the first. The second is more tokens and usually more accurate on arithmetic-heavy extraction."}]},{"t":"h2","id":"validate-anyway","runs":[{"text":"Validate anyway"}]},{"t":"p","runs":[{"text":"The schema is a syntax check. Your validator is the semantic one, and it runs in your code after the response arrives:"}]},{"t":"ol","start":1,"items":[[{"text":"Parse.","b":true},{"text":" It will parse; assert it anyway, because streaming, truncation and provider errors all produce partial documents."}],[{"text":"Types and ranges.","b":true},{"text":" A total of 4820000000 in a schema that says number is legal JSON and an obvious data error."}],[{"text":"Cross-field arithmetic.","b":true},{"text":" Line items should sum to the total. Say so in code."}],[{"text":"Provenance.","b":true},{"text":" Every number and identifier should be findable in the source text you sent. If it is not there, the model produced it, and you should know that before it reaches a ledger."}]]},{"t":"h2","id":"the-free-path","runs":[{"text":"The free path"}],"role":"free"},{"t":"p","runs":[{"text":"You do not need a provider’s structured-output feature. "},{"text":"Outlines","b":true},{"text":", "},{"text":"llama.cpp","b":true},{"text":" with a GBNF grammar, and "},{"text":"guidance","b":true},{"text":" all do constrained generation against local models, and "},{"text":"jsonschema","code":true},{"text":" or "},{"text":"Pydantic","b":true},{"text":" in Python validates whatever comes back regardless of who generated it. Pydantic is worth learning on its own: the model class doubles as the schema you send and the validator you run on return, which keeps the two from drifting apart: a drift that produces the worst kind of bug, where the contract you enforce and the contract you asked for quietly differ."}]}],"check":{"question":[{"text":"An extraction schema marks "},{"text":"due_date","code":true},{"text":" as a required string, constrained decoding is on, and a quarter of the invoices have no due date printed anywhere. What happens on those invoices, and why?"}],"options":[{"text":[{"text":"The model emits a plausible date, because the sampler is masked to tokens that keep the object valid and a missing required field is not valid."}],"correct":true,"tempting":null},{"text":[{"text":"The response fails schema validation and the harness can retry or fall back."}],"correct":false,"tempting":[{"text":"Assumes the constraint is checked after generation; it is applied during generation, so an invalid object is never produced to be caught."}]},{"text":[{"text":"The model returns an empty string, which is the conventional signal for a missing value."}],"correct":false,"tempting":[{"text":"An empty string is legal under the schema, but nothing pushes the model towards it: training data is full of complete invoices, so it fills."}]},{"text":[{"text":"The field is left out, since the model has no evidence to support any value."}],"correct":false,"tempting":[{"text":"Omission is precisely what “required” forbids, and the mask enforces the requirement token by token."}]}],"order":[1,0,3,2],"why":null},"prev":{"slug":"tools-and-function-calling","title":"A tool call is a request, not an action","minutes":7},"next":{"slug":"the-message-list","title":"What the model actually sees each turn","minutes":8}}}