Every field in the response, and the two that catch bugs

Most people read one field

A typical first integration reaches straight for the text: response.content[0].text, or response.choices[0].message.content, and everything else in the object is ignored. That is where a specific class of production bug is born, because two of the discarded fields are the ones that tell you the answer is incomplete.

Here is what comes back, and what each part is for.

id: a unique identifier for this generation. Log it. When a user reports a bad answer three days later, the id is the only reliable way to find that exact call in the provider’s dashboard and in your own traces.

model: the model that actually served the request, which is not always the string you sent. Ask for an alias like latest and you get whatever the provider is currently pointing that alias at. Log the returned value, not the requested one. The morning your outputs shift for no reason you changed, this field is the evidence.

role: always assistant. Present for symmetry with the messages you sent.

content: a list of blocks, not a string. A response can contain a text block, then a tool-use block, then more text. Any code written as content[0].text is one tool call away from a crash or, worse, from silently dropping half the answer. Iterate over blocks and switch on the type.

stop_reason (or finish_reason): why generation ended. This is the field that catches bugs.

usage: token counts. This is the field that controls your bill.

stop_reason, value by value

• end_turn / stop: the model finished naturally. This is the only value that means the answer is complete.
• max_tokens / length: it ran out of budget mid-sentence. The text you have is truncated. If you were expecting JSON, you have half an object. Parsing it will throw, or, if you were unlucky enough to be building a string, it will succeed with missing fields.
• tool_use / tool_calls: it wants you to run something and come back. Not an answer.
• stop_sequence: it hit a string you told it to stop at.
• refusal or a content-filter reason: it declined. Some providers return this as a normal completion with empty text, which is why blindly reading content[0].text can hand a user a blank screen with no error anywhere in your logs.

The rule, in one line: check stop_reason before you touch the text.

if r.stop_reason == "max_tokens":
    log.warning("truncated", extra={"id": r.id})
    raise Truncated()      # retry with a larger cap, do not parse
if r.stop_reason == "tool_use":
    return handle_tools(r)

The most common form of this bug: extraction works for weeks, then a customer submits an unusually long invoice, the response hits max_tokens, the JSON is truncated, your parser throws, and your error message says Unexpected end of JSON input, which tells you nothing about the actual cause. One if at the top would have named it.

usage, and the four numbers

usage reports at minimum input_tokens and output_tokens. On providers with prompt caching you also get cache_creation_input_tokens and cache_read_input_tokens, which are billed at very different rates: cache reads are often around a tenth of the normal input price, and cache writes a little more than normal.

Write one function and call it from every code path:

def record(r, feature, user_id):
    cost = (r.usage.input_tokens  * IN_RATE
          + r.usage.output_tokens * OUT_RATE) / 1_000_000
    metrics.increment("llm.cost", cost, tags=[f"feature:{feature}"])

Without per-feature tagging, your provider dashboard gives you one number for the whole company and you cannot answer the question that always gets asked: which feature is spending the money? That question arrives about a week after launch, and reconstructing the answer from historical logs you did not write is not possible.

What is not in the response

Two absences worth naming.

There is no confidence score. Nothing in the response tells you how likely the answer is to be right. Some providers expose logprobs, which give the model’s probability for each token it chose, useful for detecting that the model was torn between two spellings, close to useless as a measure of factual correctness. A model can assign 0.99 to every token of a confidently invented citation.

There is no indication of what the model used. You cannot tell from the response whether an answer came from a document you supplied or from parameters trained two years ago. If you need that distinction (and for anything factual you do) you have to design for it by making the model quote and cite, then verifying the quote appears in your source. The response object will not do it for you.