Rows, records and schemas in the window

Not everything worth retrieving is a document

The question is “how many of my orders shipped late last month”. No amount of chunking and embedding will answer it, because the answer is a count over rows that no document contains. Somewhere between a third and a half of the questions people ask a business assistant are like this, and a pipeline built entirely around passage retrieval fails all of them in the same confident way.

Structured data enters the window through three different doors, and choosing the right one is most of the work.

Door one: the row goes in the context

Small, specific results. The customer’s order. Their current plan. Three open tickets.

Serialise them so each record stands alone:

order=ORD-88213 status=shipped placed=2026-08-14 delivered=2026-08-19
  items=2 total=4,180 INR carrier=Bluedart

Repeating the field names on every record costs tokens and buys robustness: the record survives being split, reordered or quoted in isolation. For a handful of records that trade is clearly right.

Above roughly twenty records it inverts, and a header-once format is much cheaper: the field names stop being worth their repetition. State the header immediately above the block so nothing can separate them.

Two rules regardless of format. Include units and currency on every number, because 4180 and 4,180 INR are different facts and the model will guess. And state the row count and whether the set is complete: “12 of 87 orders, most recent first”. Without it the model treats twelve as all of them, which is the same partial-set failure the previous module described.

Door two: the schema goes in the context, and the model writes a query

For aggregates and joins (counts, sums, groupings) the data cannot go in the window and should not. Put the schema in and let the model produce SQL.

What the schema block needs, in order of value:

1. Table and column names, with types.
2. A one-line description per column where the name is not self-explanatory. status is not self-explanatory.
3. Two or three example values per categorical column. This is the highest-value item and the most often omitted. A model that does not know status contains shipped, pending, cancelled will write WHERE status = 'Shipped' and get zero rows.
4. The joins that are allowed, stated explicitly. Models invent plausible foreign keys.

This block is stable, so it belongs in the cached prefix. A 60-table schema is thousands of tokens on every request; select the tables relevant to the question first, using the same retrieval machinery you use for documents, and send five tables instead of sixty.

Door three: a tool call, and only the answer comes back

Where the question maps to a known operation (order status, account balance, inventory) do not generate SQL at all. Expose a function, let the model call it with parameters, and put only the result in the window.

This is safer (no generated SQL touching your database), cheaper (no schema in context), and more reliable (the query was written by you). It is the right default for the twenty questions that make up most of your traffic. Generated SQL is for the long tail you could not anticipate.

The failure to design against

Generated SQL fails silently. A query with a subtly wrong join returns rows (plausible rows, fewer than it should) and the model reports the number without any signal that anything went wrong. There is no exception, no empty result, nothing to catch.

Defences, in order of effectiveness:

• Run it read-only, as a role with SELECT on specific views and nothing else. Not “instruct the model not to write”. A database permission.
• Return the query alongside the answer, and show it in the interface. A user who knows their data spots a wrong filter instantly, and this is the single most effective check available.
• Assert on the shape. Zero rows where rows were expected, a count larger than the table, a date outside the plausible range: cheap assertions that catch the loud failures.
• Cap and time-limit every generated query. LIMIT and a statement timeout, enforced by the database.

Spreadsheets are not tables

A worksheet a human maintains has merged cells, three header rows, a totals row in the middle, colour that carries meaning, and notes in column M. Reading it with openpyxl and serialising the cells produces confident nonsense.

The honest answer is that messy spreadsheets need a human-specified mapping (which rows are headers, which columns matter, what a blank means) and that this is per-file work. Automating it is a research problem, not an afternoon. Where it matters, build the importer for the specific file shape and validate it; where it does not, converting the sheet to a clean CSV by hand is a legitimate engineering decision rather than a defeat.