Deprecations and the pinning problem

Models retire

Every provider retires models. The notice is typically six to twelve months, occasionally less, and on the date the endpoint returns an error. Each provider publishes a page listing what retires when. Almost nobody reads it until the error arrives, usually at an inconvenient hour, in production, on the feature nobody has touched in a year because it was working.

Aliases and snapshots

Providers offer two kinds of model name. A dated snapshot (something like model-2026-03-15) is a fixed set of weights that behaves the same today and next month. An alias (model-latest, or an undated name) resolves to whatever the provider currently considers that model, and changes when they ship a new version.

An alias changes behaviour silently. The new version refuses slightly different things, formats JSON slightly differently, writes longer answers, calls tools with a different style. The harness goes red on a Tuesday with no commit; or, if nobody ran it, the feature degrades quietly and the first sign is a complaint. The shipping lesson made model changes go through the same process as prompt changes; an alias bypasses that process by design.

Pin in production, upgrade on purpose

Production uses a dated snapshot. So does the harness, so that a red run means something changed on your side. The upgrade to a newer snapshot is a change: run the harness on the candidate, fix the prompt divergences with per-model overrides, shadow it, canary it, flip the flag. Record the pinned snapshot’s retirement date in the same configuration file as its name, so the two are never separated.

The calendar

Once a quarter, someone reads the deprecation pages for every provider in use and, for each pinned snapshot, writes down the months remaining. Under three months, the migration starts. That is enough time for the harness to show what broke, for the prompt fixes, for a week of shadow and a week of canary, with slack for the surprise that always turns up.

The cost of skipping it

The migration happens anyway, on the provider’s date, under pressure. The harness fails on a fifth of cases against the replacement model, the prompt fixes are made in an afternoon without shadow or canary, and the product ships a behaviour change nobody evaluated. Or the migration does not happen, and the endpoint returns an error at two in the morning, and the fallback chain from the last lesson (if it exists) is what the users get for a week.

Keep prompts portable where it is cheap

Some prompts lean on one model’s quirks: a particular tag format it happens to emit, a refusal phrase the code matches by string, an assumption that it always returns a list in a certain shape. Each of these is a migration cost waiting to be paid. Prefer structured output over parsing prose. Show the format with an example rather than assuming the model knows it. Check stop reasons and usage through the abstraction layer’s normalised fields rather than provider-specific strings. None of this makes migration free; it makes it a week rather than a month.

Embedding models retire too, and that is worse

When a chat model retires, you change a name and re-run the harness. When an embedding model retires, every vector in the index was produced by it and is incompatible with any replacement. The retrieval module told you to store the model name with each vector and to keep the raw chunk text: this is the day that pays off. The migration is a full re-embed (five million tokens at ten cents on a hosted model, or a few hours on a CPU) followed by a re-run of the retrieval gold set, because recall will have moved. Budget it, and schedule it before the date, not on it.

Fine-tunes are pinned to their base

A model you fine-tuned sits on a base model, and when the base retires, the fine-tune goes with it. Keep the training data and the recipe in version control, so retraining on the new base is a job rather than an archaeology project.

Where the list lives

The one-page feature record from the safety module already lists model versions. Add the retirement date beside each. The quarterly check is then a read of that page against the providers’ pages, ten minutes, and the difference between an upgrade you planned and one that was done to you.