The dependency you do not control

Your prompt is not portable

Every decision you have measured in this course (how many examples, where to repeat the instruction, whether the schema costs accuracy, what k should be) was measured against one model. Those findings are properties of the pair, not of the prompt.

This is unlike almost everything else in software. A function behaves the same on a new machine. A prompt does not behave the same on a new model, and it does not fail loudly when it stops working. It gets a bit worse.

Pin the dated version

Providers offer moving aliases: names ending in -latest or with no date. They are convenient and they are a moving target, which means your production behaviour can change on a day you deployed nothing.

Pin the dated identifier in your configuration object. Then the model is a version you chose, upgraded when you decide, tested before it moves.

And log what came back rather than what you asked for. Responses report the version actually served, and that is the field worth keeping, because it is the one that can surprise you.

Pinning reduces drift rather than eliminating it

A pinned version is stable in the ways the provider controls and not in every way.

The weights stay fixed. What can still change underneath: the serving stack, quantisation or hardware, default sampling parameters, safety filtering layers, and the chat template that renders your messages into tokens. Several of these are implementation details the provider is under no obligation to announce, and changelogs are not exhaustive.

This is not an accusation of bad faith. It is the ordinary condition of depending on a hosted service, and the response is the ordinary one: do not assume stability, measure it.

Watch drift as a trend

Run the golden set on a schedule against your pinned configuration. Weekly is plenty. Plot the score over time.

The cost is small: a few hundred items, mostly mechanical checks, a batch endpoint overnight, a few dollars. What you get is the ability to distinguish “the model changed” from “our corpus changed” from “traffic changed”, which is otherwise a difficult argument to settle and one that consumes a great deal of a team’s time.

A flat line for months is a quiet and valuable thing to have. A step change on a date, with no deploy of yours, is a fact you can take to a support ticket.

Migrating, in the order that works

When a version is deprecated or a better one appears:

1. Run the golden set on the new model with the existing prompt, before changing anything. This is the baseline for the migration and it is the step most often skipped.
2. Expect it to be worse in places. Your prompt is tuned for the old model: its examples, its repetitions, its warnings all address that model’s failure modes. A newer model frequently does not need several of them and may be mildly harmed by them.
3. Re-run the ablation sweep. This is where it pays off most. Blocks that were load-bearing often stop being so, and you can usually delete material: a real reduction in cost and complexity that only a sweep will find.
4. Re-check the budget arithmetic. A different model family means a different tokeniser, so the same text is a different number of tokens. Your carefully packed budget may no longer fit, or may have room you are not using. Re-count rather than assuming.
5. Re-run the injection corpus. Robustness to injection differs between models and between versions, sometimes substantially, and it is not something release notes usually quantify.
6. Shadow, then canary, as with any other context change.

Budget real time for this. A model migration presented as a one-line configuration edit is a one-line edit followed by two weeks of re-tuning, and pretending otherwise is how migrations acquire their reputation.

Deprecation is a planning problem

Providers retire versions with notice. If your pinned version has an end-of-life date, that date belongs in the calendar with a migration window before it, not in an email somebody read once.

The failure this prevents is specific and common: a team pins a version, enjoys two years of stability, and is then forced to migrate at short notice with no golden set, no ablation history and nobody left who remembers why the prompt says what it says. Every practice in this module is what makes that migration a week’s work instead of a rewrite.

The position

You do not control this dependency and you cannot make it behave like a library. What you can do is know when it moves, know what moving costs you, and keep the instruments that let you re-tune quickly.

That is a weaker guarantee than a version-pinned package, and it is the one actually on offer. Building as though it were stronger is the mistake; building as though nothing can be relied upon is the overcorrection. Measure, pin, and keep the golden set warm.