Building the break-even model

Stop arguing and write the function

Every preceding lesson contributed one input. This one assembles them into a single number that can be defended, challenged and corrected: the monthly token volume above which running it yourself is cheaper.

The structure is two cost curves. The hosted curve passes through the origin and rises with volume. The local curve starts high (you pay the fixed costs at volume zero) and rises barely at all, because the marginal cost of one more token on hardware you already have is a fraction of a watt-hour. They cross once. That crossing point is the whole answer, and it is a volume rather than a verdict.

def local_monthly(card_price, resale, months_held, watts, tariff,
                  duty_cycle, setup_hours, hourly_rate, months_amortise_time=12):
    capital = (card_price - resale) / months_held
    hours_running = 730 * duty_cycle          # 730 hours in an average month
    power = (watts / 1000) * tariff * hours_running
    idle = (90 / 1000) * tariff * (730 - hours_running)   # idle draw, still billed
    people = (setup_hours * hourly_rate) / months_amortise_time + 2 * hourly_rate
    return capital + power + idle + people

def tokens_per_month(peak_tps, duty_cycle):
    return peak_tps * 730 * 3600 * duty_cycle / 1e6       # in millions

def break_even(local_cost, api_price_per_million):
    return local_cost / api_price_per_million             # millions of tokens

The 2 * hourly_rate term is two hours a month of maintenance. Change it if your experience differs, but do not delete it.

A worked case, with every assumption visible

A used card at 700, expected resale 350, held 36 months. 500W under load, 90W idle, tariff 0.20. Duty cycle 8%, which is a team of several people plus some overnight batch work. Peak 1,200 tokens per second. Thirty setup hours and two maintenance hours a month at a rate of 25.

• Capital: (700 − 350) / 36 = 9.7 per month
• Power under load: 0.5 × 0.20 × 58 hours = 5.8
• Idle power: 0.09 × 0.20 × 672 hours = 12.1
• People: (30 × 25) / 12 + 2 × 25 = 62.5 + 50 = 112.5
• Total: about 140 per month, of which 80% is the person.

Tokens produced: 1200 × 730 × 3600 × 0.08 ÷ 1e6 ≈ 252 million a month, giving about 0.56 per million tokens.

Against a hosted model at, say, 0.50 per million blended, break-even is 140 / 0.50 = 280 million tokens a month. The deployment produces 252 million, so on these assumptions it is marginally behind; and it becomes clearly ahead the moment you either raise the duty cycle or stop counting the person’s hours.

That result is the honest one for a great many small deployments, and it is why the conclusion of this module is not “local is cheaper”.

Which input to argue about

Vary each input by a factor of two and see what happens to the answer. In almost every configuration the order is the same:

1. Duty cycle: moves the result nearly proportionally. Dominant.
2. Whether people’s hours are counted: often changes the sign of the conclusion by itself.
3. Amortisation window: meaningful, and entirely a choice.
4. Card price and resale: meaningful but bounded.
5. Electricity tariff: small when busy, and mostly showing up as idle draw.

So a disagreement about the electricity price is not worth the meeting, and a disagreement about duty cycle is the only conversation that matters. Ask for the measurement.

The reality check on the break-even volume

Translate the crossing point into something human before presenting it. 280 million tokens a month is roughly 9.3 million a day. At around 700 tokens per exchange (a question, some context and an answer) that is about 13,000 exchanges a day, every day.

Write that sentence down and ask whether it describes your organisation. For a team of ten people using a chat interface, it does not, and no arrangement of the hardware will make it. For a pipeline classifying every incoming support message, enriching a product catalogue, or generating embeddings over an archive, it is easily exceeded, and those are precisely the tasks a small local model does well.

What the model does not capture

Be explicit about this when you present it, because someone will otherwise use the number as though it settled everything. The model prices tokens. It does not price data that cannot leave the building, a regulatory requirement, working without connectivity, a fixed and predictable cost where a metered one is hard to approve, or the value of the skill acquired. Those are real reasons that justify local at any volume, and they should be argued on their own terms rather than smuggled into the arithmetic.

Present the break-even as a volume and a list of things it does not count. A single number presented as an answer will be believed, and then blamed.