What a training run actually costs

The arithmetic, and why it is worth doing

Training costs are quoted as impressive numbers with no working. You can do the working yourself, and the result changes how you read announcements.

Training compute ≈ 6 × N × D, where N is parameters and D is training tokens. The 6 breaks down as roughly 2 operations per parameter for the forward pass and 4 for the backward pass.

Take an 8B model on 15 trillion tokens:

6 × 8e9 × 15e12 = 7.2e23 floating-point operations

Now convert to time. A modern accelerator might deliver a few hundred teraflops of sustained bf16 throughput on a real workload: well below its headline peak, since utilisation of 35 to 50 per cent is considered good. At 400 teraflops sustained:

7.2e23 / 4e14 = 1.8e9 seconds = about 500,000 accelerator-hours

On 1,000 accelerators, roughly three weeks. At cloud rental rates of a few dollars per accelerator-hour, on the order of one to two million dollars of compute for the final run.

What that number leaves out

The final run is a fraction of the true cost. Add:

• Failed and abandoned runs. Loss spikes, bad hyperparameters, corrupted data shards. Several restarts is normal.
• Ablations. Small-scale experiments to choose the data mixture and architecture. Often more total compute than the final run.
• Data pipeline. Crawling, extracting, filtering and deduplicating tens of terabytes is a large engineering effort in its own right.
• Post-training. Instruction data, human preference collection, safety evaluation. Human annotation at scale is expensive in a way compute is not.
• Salaries. Frequently the largest line, and never in the compute figure.

So a headline of “a few million dollars” for a frontier training run should be read as the compute for one successful pass, not the cost of producing the model.

Inference, and where the money actually goes

Inference compute ≈ 2 × N per token. For an 8B model, 16 billion operations per generated token.

That sounds trivial next to 7.2e23. It is, per token. But a successful model serves billions of tokens a day, every day, for years. Across a deployed model’s lifetime, total inference compute typically exceeds training compute, often by a wide margin.

This is the fact behind several industry behaviours that otherwise look strange: the willingness to spend enormous effort distilling models smaller, the investment in serving optimisations from module three, and the preference for sparse architectures. All of them attack the recurring cost rather than the one-off.

Energy and the numbers people quote

Energy figures for AI circulate widely and vary by orders of magnitude, which should make you cautious about all of them.

What can be said with reasonable confidence: a training run of the scale above consumes energy comparable to a few hundred households for a year, and the per-query energy of a small text request is small: Google published a median figure of about 0.24 watt-hours per text prompt for its assistant in 2025, roughly nine seconds of a television. Multiply by billions of queries and the aggregate is substantial; the per-query figure is not the scary part.

Two honest caveats. Published figures come from the companies with an interest in them and use methodologies that differ on what to include. And reasoning models that generate thousands of internal tokens per answer cost far more per query than the simple case, so a median figure from one product tells you little about another.

Using this

When you read that a model cost a certain amount, check whether the number is consistent with 6ND and public parameter counts. When a vendor quotes a price per million tokens, compare it against 2N and current hardware costs to see whether it is plausibly above cost or subsidised. And when someone quotes an energy figure, ask what it includes.

None of this requires inside information. It requires two formulas and public numbers, and it is the difference between reading these announcements and evaluating them.