Structured answers you can render

Text is not enough

A summary can be shown as a paragraph. A category, a list of action items with owners and dates, a score out of ten, a set of tags: these have to be read by code, and code cannot read prose. The model has to answer in a shape.

The generated version asks “respond in JSON”, runs JSON.parse on whatever comes back, and works until the day the model wraps the JSON in a markdown fence, adds a sentence before it, or returns a field the code did not expect. Each of those is a crash in a route that had been fine for weeks.

Declare the shape

Every major provider now has a way to say what shape you want and have the model held to it: JSON mode, a response schema, or a tool definition whose arguments are your structure. The exact names differ; the idea is the same and the compatibility layers pass it through:

response_format: {
  type: "json_schema",
  json_schema: {
    name: "expense",
    schema: {
      type: "object",
      properties: {
        category: { type: "string", enum: ["travel", "food", "equipment", "other"] },
        amount: { type: "number" },
        confidence: { type: "number", minimum: 0, maximum: 1 },
      },
      required: ["category", "amount", "confidence"],
      additionalProperties: false,
    },
  },
}

With a schema, the model’s output is constrained during generation on providers that support it, and you get valid JSON in that shape nearly every time. Nearly.

Validate anyway

The schema is a request to the provider. Your code’s assumptions are your responsibility. Parse and validate before anything reads the result:

import { z } from "zod";

const Expense = z.object({
  category: z.enum(["travel", "food", "equipment", "other"]),
  amount: z.number().nonnegative(),
  confidence: z.number().min(0).max(1),
});

const raw = data.choices[0].message.content ?? "";
const parsed = Expense.safeParse(JSON.parse(stripFences(raw)));
if (!parsed.success) { /* the retry below */ }

stripFences removes a leading fence line (three backticks and json) and a trailing fence if present; models add them even when told not to, and this is cheaper than arguing. safeParse returns a result rather than throwing, and the failure carries the exact reason (“category: expected one of travel | food | equipment | other, received Miscellaneous/Other”) which is the one thing you want in the log and the one thing the retry needs.

Retry once, with the error

On a validation failure, send one more request with the original prompt plus the error message: “Your previous answer failed validation: <error>. Reply again with only the JSON.” That fixes most cases, because the model is good at correcting a specific mistake it can see.

Once. A loop that retries until valid can run for minutes on a model that has decided the category is “Miscellaneous/Other”, and each attempt costs tokens.

The second failure needs a plan

After the retry, the answer is still invalid. What now depends on the feature, and deciding it in advance is the design work:

• A category can fall back to other and be flagged for a person to look at.
• A score can be recorded as missing rather than as zero, because zero is a number and the page will average it.
• A list of action items can be shown as the raw text with a note that it could not be structured.

What never happens is silently substituting a default and moving on as if the model had said it. A missing value is a fact about this input; a fabricated one is a lie the rest of the app will repeat.

Numbers, dates and the enum it invents

Three shapes cause most of the residual failures. Numbers arrive as strings ("amount": "42.50"), and z.coerce.number() handles it. Dates arrive in whatever format the input used, and the only safe request is ISO 8601 with an explicit instruction and a validation that parses it. And enums grow: told four categories, the model will eventually produce a fifth that is a plausible blend of two. The enum in the validator is what catches it; the prompt is what makes it rare.

What structured output cannot do

It cannot make the content correct. A well-formed {"amount": 4250} for a receipt that says 42.50 passes every check above. The shape is guaranteed; the values are still the model’s reading of the input, and the lesson on whether the feature is good is where that gets measured.