What it costs to keep it alive

The part nobody budgets for

Building an automation is a project. Owning one is a subscription, and the currency is attention.

Here is what changes underneath a working system, none of which is your fault:

• An upstream API changes. A field is renamed, a limit is tightened, an endpoint is deprecated. Expect this a few times a year per integration.
• A model is retired. Providers publish deprecation dates; the failure mode is that nobody reads the email. Pin your model version, and put the retirement date in a calendar when you pin it.
• Prices change, and the economics you calculated in module one no longer hold.
• A business rule changed and nobody told the automation. This is the most common and the least visible, because nothing errors. The system keeps applying last year’s discount policy with perfect reliability.
• The person who understood it left.

The quarterly review, twenty minutes

Once a quarter, for every automation, answer six questions from data you already log:

1. Is it still running?
2. What is the volume, and is it changing?
3. What is the error rate, and the validator rejection rate?
4. What did it cost last quarter?
5. Who owns it now, and are they still here?
6. Does anyone read the output?

Twenty minutes with a spreadsheet. The first four come from the run log you built in module one. The fifth is a name. The sixth kills more automations than the other five together, and you should ask it first.

Deleting one is a good outcome

Somewhere in your organisation is a beautifully engineered weekly report with an open rate of zero. It costs money every week, it breaks twice a year, and somebody spends a day fixing it each time because it is “in production”.

Check the open rate before you optimise the pipeline. Turning something off is a legitimate, unglamorous, high-return engineering result, and the way to make it easy is to leave it off for a month and see who asks. Frequently nobody does.

Handing it over

A new owner needs four things:

• the runbook, including the failure history
• the golden set, so they can change something and know if they broke it
• access to the traces, so they can see what happened rather than guess
• one real run, walked through together, from trigger to output

That last one is not optional and takes an hour. If you cannot hand an automation over in about two hours, it is too clever for its job, and the excess cleverness is a liability with a person’s name on it.

The honest closing position

The automations that survive years are small, boring, observable, owned by a named person, and reviewed on a schedule. They have one model call where a model is needed, deterministic code everywhere else, a validator, an alert that fires when nothing happens, and a runbook with real failures in it.

Almost none of that is specific to AI, which is exactly the point. The model was never the hard part. Keeping a system honest, in contact with a world that keeps changing, is the hard part, and it always was.