When running it yourself is the wrong answer

The enthusiasm and the correction

Running a model on your own hardware is satisfying and useful, and it is oversold by people who enjoy it. This lesson is the correction, written by somebody who thinks the previous ones in this module are right.

Seven cases where local is the wrong choice

You need the hardest reasoning. A model that fits your laptop is roughly a year behind the frontier on difficult multi-step problems, and the gap is largest on exactly the tasks where you would most value help.

Your time is worth more than the difference. Setup is an evening. Keeping up with new releases, formats and runner updates is an hour a month if you care. If a $20 subscription is not a strain, that arithmetic frequently favours paying.

You are buying hardware to save on subscriptions. A graphics card costs the equivalent of two to four years of a subscription, will be superseded, and only fits the model sizes it fits. If you want the card anyway, that is a different decision. As a cost-saving measure it usually is not one.

You need the whole product. Web search, file handling, memory, a mobile app, voice, sharing with colleagues. You can assemble these around a local model and it is a project, not a setting.

You are on a laptop battery. Local inference is a heavy, sustained load. It warms the machine, spins the fans, and shortens the working day considerably.

You need it to work for other people. The moment a second person depends on it, you have become the operator of a service: availability, updates, security patches, and someone to blame when it is down.

You are relying on a safety layer. A downloaded model arrives with whatever safety training its makers gave it and no product layer around it. If it will be used by children, students or the public, that absence is your responsibility to fill.

The security surface people forget

A self-hosted front-end reachable from the internet is a service you are running, and it needs the same care as any other. The failure pattern is dull and common: an instance put up quickly for convenience, left unpatched and without a login, holding both an API key and a year of documents.

If you run one, keep it on your own machine or behind authentication, keep it updated, and do not put anything on a public address without deciding deliberately to run a public service.

Where local is unambiguously right

To be balanced, the cases where nothing else will do:

• The material must not leave the building, as a matter of law, contract or professional duty.
• There is no reliable connection.
• The volume is enormous and the task is routine.
• You need the same behaviour in five years with no deprecation notice.
• You are learning how these systems work, which is a good reason on its own.
• You cannot pay, and this is the route that costs nothing after the download.

The maintenance nobody mentions

There is an ongoing cost that never appears in a setup guide. Runners update and occasionally change behaviour. Model formats evolve, and a file that worked last year may need re-downloading in a new format. Better models come out constantly, so the model you installed is quietly becoming the wrong choice. And an operating system upgrade can break a GPU driver stack in ways that take an evening to unpick.

None of this is difficult. It is an hour a month if you keep up and a lost weekend if you do not, and it is the honest reason many people who set this up enthusiastically are back on a subscription a year later. Decide whether you want that hour before you begin, rather than discovering it afterwards.

The summary

A local model is a very good second tool and a demanding first one. The arrangement that works for most people is the one at the end of the previous lesson: a local model for the material that requires it, something hosted for everything else, and no ideological commitment to either.

The mistake in both directions is the same. Choosing on identity (“I self-host everything” or “I only use the best model”) instead of on what the particular piece of work needs.