{"course":{"slug":"context-engineering","title":"Context Engineering","url":"https://addaly.com/learn/context-engineering","level":"advanced","track":"build"},"lesson":{"slug":"forcing-a-schema","title":"Structured output, and what a schema does not buy you","url":"https://addaly.com/learn/context-engineering/forcing-a-schema","position":58,"of":87,"minutes":9,"answer":"A schema constrains the shape of an answer, never its truth, and its field order is a prompt.","keyIdea":"A schema constrains the shape of an answer, never its truth, and its field order is a prompt."},"body":{"format":"markdown","text":"## Three levels of forcing\n\n**Asking.** \"Reply with JSON only.\" Works most of the time, fails on the call where the model opens with \"Here's the JSON:\" or wraps it in a fence or trails an apology. At 100,000 calls a day, a 0.3% failure rate is 300 broken records.\n\n**Tool calling.** You declare a function with a JSON Schema; the model emits arguments. Better, because the model was trained on this shape, but most implementations still validate after the fact rather than during generation.\n\n**Constrained decoding.** At each step the sampler masks every token that cannot appear next in a valid document, so invalid tokens have probability zero. This is what JSON-schema modes, grammar-based sampling (GBNF in llama.cpp), and libraries like Outlines, XGrammar and llguidance actually do. The output *cannot* be malformed. Not \"almost never\": cannot.\n\nSo the parser problem is solved. That is the whole of what it solves.\n\n## The part the schema does not do\n\nConstrained decoding shapes tokens. It knows nothing about the world. `{\"invoice_total\": 0, \"currency\": \"INR\"}` is perfectly valid and perfectly wrong. Teams turn on strict mode, watch their crash rate go to zero, and report that accuracy \"improved\": what improved is that failures stopped being visible as exceptions and started being visible as quiet nonsense in the database.\n\nWorse, a tight constraint can *reduce* accuracy. If your enum is `[\"refund\", \"billing\", \"technical\"]` and a ticket is about account deletion, the model is forbidden from saying so. It must pick one. You have engineered a guaranteed wrong answer.\n\nAlways include an escape member:\n\n```json\n{\n  \"category\": {\n    \"type\": \"string\",\n    \"enum\": [\"refund\", \"billing\", \"technical\", \"account\", \"unknown\"]\n  }\n}\n```\n\nThe same goes for every extracted field: `null` must be reachable, and your prompt must say that \"not stated in the document\" is a correct answer rather than a failure.\n\n## Field order is prompt engineering\n\nGeneration is left to right. The model emits your fields in schema order, and each field is conditioned on the ones before it. So this schema:\n\n```json\n{ \"label\": {...}, \"reasoning\": {\"type\": \"string\"} }\n```\n\nproduces the label first and *then* a justification for a decision already made. The reasoning is decoration. Reverse it:\n\n```json\n{\n  \"quoted_evidence\": {\"type\": \"string\"},\n  \"reasoning\":       {\"type\": \"string\"},\n  \"label\":           {\"type\": \"string\", \"enum\": [...]},\n  \"confidence\":      {\"type\": \"string\", \"enum\": [\"high\", \"medium\", \"low\"]}\n}\n```\n\nand the label is now conditioned on evidence the model had to commit to in writing. This is one of the highest-return changes available in a structured pipeline and it costs you a schema edit. Asking for a quotation first is stronger than asking for reasoning first, because a quotation is checkable: you can assert in code that the string actually appears in the source.\n\n## Practical constraints worth knowing\n\n- **Providers support different subsets.** Recursion, `pattern`, `minimum`/`maximum`, and `oneOf` are unevenly supported and sometimes silently dropped. Validate after generation regardless of what strict mode promises.\n- **Depth and unions hurt.** A schema with six levels of nesting and a twelve-way union produces worse content than a flat one, even when both are structurally valid. Flatten; run two calls if you must.\n- **Descriptions are read.** Field descriptions land in the prompt. `\"amount_minor_units\": \"integer, in the smallest unit — 1500 for ₹15.00, 1500 for $15.00\"` does real work.\n- **Streaming is awkward.** Partial JSON is not JSON. Use a tolerant incremental parser, or stream a text field and structure the rest at the end.\n- **Retries need the error.** On validation failure, feed the validator message back rather than saying \"that was wrong\".\n\n```python\nfor attempt in range(3):\n    raw = call(messages, schema=SCHEMA)\n    try:\n        return Model.model_validate_json(raw)\n    except ValidationError as e:\n        messages += [\n            {\"role\": \"assistant\", \"content\": raw},\n            {\"role\": \"user\", \"content\": f\"Validation failed:\\n{e}\\nReturn corrected JSON only.\"},\n        ]\nraise ExtractionFailed()\n```\n\n## Where it fits\n\nSchemas are how you turn a language model into a component with a type signature. Everything downstream (routing, evaluation, retries, storage, the injection defences in Lesson 8) gets easier once the output has a shape you can assert on. Just keep the two claims separate in your head: the schema guarantees the shape, and nothing guarantees the truth."},"check":{"question":"A classifier uses strict schema mode with the fields declared in this order: `label` (enum), then `reasoning` (string). The reasoning text is consistently fluent and consistently defends the label, including when the label is wrong. What is happening?","options":[{"text":"The reasoning is generated after the label and conditioned on it, so it can only rationalise a decision that has already been made.","correct":true},{"text":"JSON objects are unordered by specification, so the model plans both fields together and key order carries no meaning.","correct":false,"tempting":"The JSON spec really does treat objects as unordered, which makes this feel like a correctness argument, but decoding is strictly sequential over the emitted characters."},{"text":"The model reads the full schema before generating, so it has already worked out the reasoning internally by the time it emits the label.","correct":false,"tempting":"The schema is visible up front, so it seems the plan could precede output, but reasoning that is not written down does not condition later tokens the way emitted text does."},{"text":"Temperature is too high; at temperature 0 the label and its justification would become consistent.","correct":false,"tempting":"Sampling noise does cause inconsistency in other settings, so lowering temperature is a habitual fix, but it would make a wrong label more stable rather than better justified."}]},"prev":{"slug":"telling-the-user-it-forgot","title":"Being honest about a bounded window","url":"https://addaly.com/learn/context-engineering/telling-the-user-it-forgot"},"next":{"slug":"how-a-schema-is-enforced","title":"How a schema is actually enforced","url":"https://addaly.com/learn/context-engineering/how-a-schema-is-enforced"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Structured output, and what a schema does not buy you', in Context Engineering. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/context-engineering/forcing-a-schema (Accessed: 31 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Structured output, and what a schema does not buy you","courseTitle":"Context Engineering","module":{"index":7,"slug":"tools-schemas-and-the-agent-window","title":"Tools, schemas and the agent’s window"},"answer":null,"keyIdea":"A schema constrains the shape of an answer, never its truth, and its field order is a prompt.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"three-levels-of-forcing","runs":[{"text":"Three levels of forcing"}]},{"t":"p","runs":[{"text":"Asking.","b":true},{"text":" “Reply with JSON only.” Works most of the time, fails on the call where the model opens with “Here’s the JSON:” or wraps it in a fence or trails an apology. At 100,000 calls a day, a 0.3% failure rate is 300 broken records."}]},{"t":"p","runs":[{"text":"Tool calling.","b":true},{"text":" You declare a function with a JSON Schema; the model emits arguments. Better, because the model was trained on this shape, but most implementations still validate after the fact rather than during generation."}]},{"t":"p","runs":[{"text":"Constrained decoding.","b":true},{"text":" At each step the sampler masks every token that cannot appear next in a valid document, so invalid tokens have probability zero. This is what JSON-schema modes, grammar-based sampling (GBNF in llama.cpp), and libraries like Outlines, XGrammar and llguidance actually do. The output "},{"text":"cannot","i":true},{"text":" be malformed. Not “almost never”: cannot."}]},{"t":"p","runs":[{"text":"So the parser problem is solved. That is the whole of what it solves."}]},{"t":"h2","id":"the-part-the-schema-does-not-do","runs":[{"text":"The part the schema does not do"}]},{"t":"p","runs":[{"text":"Constrained decoding shapes tokens. It knows nothing about the world. "},{"text":"{\"invoice_total\": 0, \"currency\": \"INR\"}","code":true},{"text":" is perfectly valid and perfectly wrong. Teams turn on strict mode, watch their crash rate go to zero, and report that accuracy “improved”: what improved is that failures stopped being visible as exceptions and started being visible as quiet nonsense in the database."}]},{"t":"p","runs":[{"text":"Worse, a tight constraint can "},{"text":"reduce","i":true},{"text":" accuracy. If your enum is "},{"text":"[\"refund\", \"billing\", \"technical\"]","code":true},{"text":" and a ticket is about account deletion, the model is forbidden from saying so. It must pick one. You have engineered a guaranteed wrong answer."}]},{"t":"p","runs":[{"text":"Always include an escape member:"}]},{"t":"code","lang":"json","label":"JSON","text":"{\n  \"category\": {\n    \"type\": \"string\",\n    \"enum\": [\"refund\", \"billing\", \"technical\", \"account\", \"unknown\"]\n  }\n}"},{"t":"p","runs":[{"text":"The same goes for every extracted field: "},{"text":"null","code":true},{"text":" must be reachable, and your prompt must say that “not stated in the document” is a correct answer rather than a failure."}]},{"t":"h2","id":"field-order-is-prompt-engineering","runs":[{"text":"Field order is prompt engineering"}]},{"t":"p","runs":[{"text":"Generation is left to right. The model emits your fields in schema order, and each field is conditioned on the ones before it. So this schema:"}]},{"t":"code","lang":"json","label":"JSON","text":"{ \"label\": {...}, \"reasoning\": {\"type\": \"string\"} }"},{"t":"p","runs":[{"text":"produces the label first and "},{"text":"then","i":true},{"text":" a justification for a decision already made. The reasoning is decoration. Reverse it:"}]},{"t":"code","lang":"json","label":"JSON","text":"{\n  \"quoted_evidence\": {\"type\": \"string\"},\n  \"reasoning\":       {\"type\": \"string\"},\n  \"label\":           {\"type\": \"string\", \"enum\": [...]},\n  \"confidence\":      {\"type\": \"string\", \"enum\": [\"high\", \"medium\", \"low\"]}\n}"},{"t":"p","runs":[{"text":"and the label is now conditioned on evidence the model had to commit to in writing. This is one of the highest-return changes available in a structured pipeline and it costs you a schema edit. Asking for a quotation first is stronger than asking for reasoning first, because a quotation is checkable: you can assert in code that the string actually appears in the source."}]},{"t":"h2","id":"practical-constraints-worth-knowing","runs":[{"text":"Practical constraints worth knowing"}]},{"t":"ul","items":[[{"text":"Providers support different subsets.","b":true},{"text":" Recursion, "},{"text":"pattern","code":true},{"text":", "},{"text":"minimum","code":true},{"text":"/"},{"text":"maximum","code":true},{"text":", and "},{"text":"oneOf","code":true},{"text":" are unevenly supported and sometimes silently dropped. Validate after generation regardless of what strict mode promises."}],[{"text":"Depth and unions hurt.","b":true},{"text":" A schema with six levels of nesting and a twelve-way union produces worse content than a flat one, even when both are structurally valid. Flatten; run two calls if you must."}],[{"text":"Descriptions are read.","b":true},{"text":" Field descriptions land in the prompt. "},{"text":"\"amount_minor_units\": \"integer, in the smallest unit — 1500 for ₹15.00, 1500 for $15.00\"","code":true},{"text":" does real work."}],[{"text":"Streaming is awkward.","b":true},{"text":" Partial JSON is not JSON. Use a tolerant incremental parser, or stream a text field and structure the rest at the end."}],[{"text":"Retries need the error.","b":true},{"text":" On validation failure, feed the validator message back rather than saying “that was wrong”."}]]},{"t":"code","lang":"python","label":"Python","text":"for attempt in range(3):\n    raw = call(messages, schema=SCHEMA)\n    try:\n        return Model.model_validate_json(raw)\n    except ValidationError as e:\n        messages += [\n            {\"role\": \"assistant\", \"content\": raw},\n            {\"role\": \"user\", \"content\": f\"Validation failed:\\n{e}\\nReturn corrected JSON only.\"},\n        ]\nraise ExtractionFailed()"},{"t":"h2","id":"where-it-fits","runs":[{"text":"Where it fits"}]},{"t":"p","runs":[{"text":"Schemas are how you turn a language model into a component with a type signature. Everything downstream (routing, evaluation, retries, storage, the injection defences in Lesson 8) gets easier once the output has a shape you can assert on. Just keep the two claims separate in your head: the schema guarantees the shape, and nothing guarantees the truth."}]}],"check":{"question":[{"text":"A classifier uses strict schema mode with the fields declared in this order: "},{"text":"label","code":true},{"text":" (enum), then "},{"text":"reasoning","code":true},{"text":" (string). The reasoning text is consistently fluent and consistently defends the label, including when the label is wrong. What is happening?"}],"options":[{"text":[{"text":"The reasoning is generated after the label and conditioned on it, so it can only rationalise a decision that has already been made."}],"correct":true,"tempting":null},{"text":[{"text":"JSON objects are unordered by specification, so the model plans both fields together and key order carries no meaning."}],"correct":false,"tempting":[{"text":"The JSON spec really does treat objects as unordered, which makes this feel like a correctness argument, but decoding is strictly sequential over the emitted characters."}]},{"text":[{"text":"The model reads the full schema before generating, so it has already worked out the reasoning internally by the time it emits the label."}],"correct":false,"tempting":[{"text":"The schema is visible up front, so it seems the plan could precede output, but reasoning that is not written down does not condition later tokens the way emitted text does."}]},{"text":[{"text":"Temperature is too high; at temperature 0 the label and its justification would become consistent."}],"correct":false,"tempting":[{"text":"Sampling noise does cause inconsistency in other settings, so lowering temperature is a habitual fix, but it would make a wrong label more stable rather than better justified."}]}],"order":[3,2,1,0],"why":null},"prev":{"slug":"telling-the-user-it-forgot","title":"Being honest about a bounded window","minutes":8},"next":{"slug":"how-a-schema-is-enforced","title":"How a schema is actually enforced","minutes":10}}}