The hours nobody invoices

The largest line in the first year

Every cost comparison in this module so far has been about machines. In most real deployments the largest number in the first year is a person’s time, and it is the one line that never appears in the spreadsheet.

This is not a rhetorical point. Work through what actually happened between deciding to self-host and having something colleagues rely on. A realistic first deployment, done carefully by someone competent but new to it, runs somewhere between twenty and forty hours: choosing and auditioning models, getting the build right for the hardware, discovering a chat template problem, configuring context and memory fractions, putting a gateway in front, writing a unit file, and the load test that tells you what you have. None of that is wasted and all of it is hours.

Steady state is smaller and never zero. A few hours a month for updates and small failures, plus a spike whenever an engine upgrade changes a default or a better model appears and the audition starts again.

Price those hours at whatever you would otherwise be paid, or at what you would pay a contractor. Thirty hours at any professional rate exceeds the cost of a mid-range card, and it usually exceeds a year of the API bill it replaced.

Where the hours actually go

The distribution is lopsided and worth anticipating, because the surprising items are the ones that break a schedule.

• Templates and tokenisers. The single most common time sink, and the least visible, because the failure is quiet degradation rather than an error.
• Version matrices. Driver, runtime, engine and Python packages disagreeing. Hours, with nothing to show for them.
• An upgrade that changed a default. A context length, a memory fraction, a sampling parameter. You discover it from a user report.
• A three-in-the-morning restart. Not because you must fix it then, but because somebody will ask whether it is fixed.
• Keeping up. Reading about new models, testing them, deciding not to switch. This is genuine work and it has no output that anyone sees.

When the hours are free

For a large part of this course’s readership, they are; and it should be said plainly rather than treated as an embarrassment.

If you are learning, the hours are the point. The skill transfers, it is in demand, and the alternative use of the evening was not billable. If you are a student without a card to put in a corporate expense claim, a free notebook tier and your own time is the entire viable path. If you are in a place where paying an overseas API involves a card you do not have or currency controls you cannot navigate, your time is not competing with a purchase you could actually make.

What changes is the decision rule, not the arithmetic. When the hours are free, local wins at far lower volumes. When the hours belong to an employer paying for them, count them, because the person who wrote the business case will be asked why the project took a month.

The organisational version: one person knows

The hidden cost that surfaces later is concentration. A self-hosted stack built by one enthusiastic person is a stack that stops when that person is on leave, and a liability when they resign. This is the most common reason organisations quietly move back to an API after a successful pilot: not cost and not quality, but that only one person can fix it.

The mitigations are cheap if done early and expensive if done after. Write a runbook that someone else can follow: how to restart it, how to roll back, what the health check is, where the logs are, who to call. Pin every version so the machine is reproducible. Choose boring, widely used components over a clever configuration that only its author understands. And have a second person do a restart and a rollback once, from the runbook, while the author watches without helping.

Reducing the number, concretely

The things that actually cut maintenance hours are unexciting. Pin the engine tag and the model revision, so nothing changes without you. Keep the previous version running during an upgrade so rollback is a configuration change. Automate the health check so the first person to notice a failure is a machine. Put quotas in the gateway so one caller cannot create an incident. And resist upgrading to each new model as it appears: the audition costs a day, and the improvement is frequently smaller than the cost of finding out.

A stack that needs no attention for a month is worth more than one that is 10% faster and needs a day.