What you send to the provider, and what you can actually verify

Every request is a disclosure

Look at one request as a list of what is in it. The system prompt: your product’s design, sometimes months of work. The user’s message: personal data, sometimes intimate. The retrieved chunks: your customers’ documents, your internal policies. The tool results: rows from your database. The attached image: a face, a location in its metadata, a document with a name on it. All of it leaves your infrastructure and arrives at a company you have a contract with, and none of it comes back.

That is not an argument against using a provider. It is the list you need in front of you to decide what should be in the request at all.

Read the terms, and know their limit

The major providers’ API terms, as of writing, say broadly the same things: data sent through the API is not used to train their models by default; it is retained for a limited period, commonly thirty days, for abuse monitoring; and enterprise customers can negotiate zero retention. Consumer chat products have different defaults, often with training on by default and an opt-out. The distinction matters: a feature built on the API and a colleague pasting customer data into a consumer chat window are governed by different terms.

These are contract terms. You can read them, you can hold the provider to them, and you cannot verify them. There is no instrument you can point at a provider’s data centre to confirm that a retention window was honoured. So your own privacy notice should say what is true: that messages are processed by a named provider under that provider’s terms. It should not say “your data is never stored”, because you do not know that.

Regions

Several providers offer endpoints in particular regions, so that processing happens inside the European Union, or inside India, or inside the United States. Whether you need one is a legal question that varies: EU data-protection law places conditions on transfers out of the EU; India’s payments regulator requires certain payment data to be stored in India; other sectors have their own rules. This is not legal advice; it is the question to put to someone who can give it, before the architecture is fixed.

Minimise before sending

What never leaves cannot be retained, trained on, leaked or subpoenaed. Before the request is assembled:

• Redact identifiers. Microsoft’s Presidio is free and detects names, phone numbers, email addresses and card numbers, and accepts custom recognisers for national identifiers such as Aadhaar and PAN patterns. Replace with placeholders the model can still reason about ([CUSTOMER_NAME], [PHONE]) and restore them in the response on your side if the task needs them.
• Pseudonymise. The model does not need user 8213’s real id; it needs “the customer”. Map ids to opaque tokens per request.
• Strip what the task does not use. The uploads lesson removed image metadata; the same logic applies to every field in a tool result. A model summarising an order does not need the customer’s address.

Each of these is a few lines of code. Together they change what a breach at the provider, or a subpoena to it, could reveal.

Secrets in the prompt

An API key in a system prompt, a database URL in a tool result, a credential the model was given so it could call a service: each is now in the provider’s logs for the retention period and in your traces for as long as you keep them. The model does not need credentials; your tool code does, on your side, after the model has asked. If a secret has ever been sent in a prompt, rotate it.

When data cannot leave at all

Health records, legal matters under privilege, government data, anything where a policy or a law says the data stays on your infrastructure: the answer is an open model on hardware you control. The cost and operations of that are the next module’s subject; the trade is capability and convenience against control, and for some data the trade is not optional.

Ask your provider what your customers will ask you

A data-processing agreement that names purposes, retention and security measures. The list of subprocessors: the provider’s own cloud vendor, and whoever else touches the data. Breach-notification terms. Where the data is processed. Whether your traffic is excluded from training in writing, not only on a web page. Enterprise customers will send you a questionnaire with these questions, and every one of them is a question you forward to your provider. Have the answers before the questionnaire arrives.

Your logs are the bigger exposure

In practice, the provider’s thirty-day retention is a smaller risk than your own trace store, which holds the same text for as long as you decided, under access controls you set, in a system your whole engineering team can read. The traces lesson gave the rules. The point here is proportion: the effort spent worrying about a provider’s policy is often better spent on your own retention.

Tell users in a sentence

“Your messages are sent to [provider] to generate replies. We do not use them to train models. They are deleted from our systems after [N] days.” Three sentences, each one you can stand behind. Every clause that is true and short is worth more than a page that hedges, and every clause that is not true is a liability the moment someone checks.