Renting an open model instead of running it

The route between a laptop and a frontier subscription

Several companies do nothing but serve open models over an API: you send a request naming llama-3.3-70b or qwen3-32b, they run it on their hardware, you pay per token. Prices sit well below frontier models and the quality on ordinary tasks is close enough that the difference rarely shows.

This is the most under-used route in the whole field. It covers the large space between “my laptop cannot run this” and “I am paying frontier prices for summarising”.

What you are comparing between providers

The same model name from two providers is not the same service. Five things differ and all of them are checkable before you commit.

Price per million tokens. Compare input and output separately.

Speed. Tokens per second varies enormously between providers running identical weights, because it depends on their hardware and serving stack. Some specialist providers are several times faster than others on the same model. If you are generating long documents, this is the difference between a workable tool and an irritating one.

Quantisation. Some serve at full precision, some at 8-bit, some at 4-bit, and many do not say. A provider serving a heavily quantised version at a lower price is offering a different product under the same name. Ask, or look for a published statement; the good ones publish it.

Context length served. Frequently lower than the model supports. A model advertised at 128k may be offered at 32k.

Data terms. The important one. Some providers state plainly that they do not train on or retain your inputs; some have a free tier that does both; some are quiet about it. This is the question to settle before the price question, because a cheap provider that keeps your client’s contract is not cheap.

Where this route wins

Bulk work. Classification, extraction, summarising, translation, tagging: the tasks where open models match frontier models and volume makes price decisive.

Predictable, moderate use. A few dollars a month rather than twenty.

Speed-critical work. The fastest hosted open models generate several hundred tokens per second, which is faster than any frontier model and changes what interactive tools feel like.

Trying a model before installing it. Cheaper than a 40 GB download you may not keep.

Data residency. Some providers let you choose the region your requests are processed in, which matters where regulation requires it.

Where it does not

The hardest tasks. Long multi-step reasoning, agentic tool use over many steps, and the deepest knowledge questions still favour frontier models. Test rather than assume; the gap has narrowed and it has not closed.

Confidentiality that must be structural. A promise in a policy is not the same as computation on your own machine. If the answer must be “the data never left the building”, this route is not it.

Feature parity. No memory, no web search, no file store unless you build them.

Checking a provider in ten minutes

Before committing to one, run three things. Send the same prompt to two providers serving the same model and compare the answers side by side; large differences point at different precision. Time a 500-word generation on each and note the tokens per second. And read the data page rather than the marketing page, looking specifically for the words retention, training and human review.

If the third of those cannot be answered in ten minutes, that is your answer for anything sensitive. Nothing else about the provider needs checking, because the question that mattered has already been left unanswered.

A sensible arrangement

For most people who have got this far, the arrangement that works is three-way and costs very little:

• Local model for anything confidential and anything offline.
• Hosted open model for volume and for routine work, at a few dollars a month.
• One frontier model, subscription or API, for the hard things.

The interesting part is how rarely the third is needed once the first two are set up properly. People who make this arrangement typically find that the frontier tool handles perhaps a tenth of their requests, and that the tenth is worth paying for while the other nine tenths were not.