What it cost to build one

Three costs that get reported as one

When a number is attached to a model (five million dollars, a hundred million, a billion) it is almost always one of three different quantities, and the reporting rarely says which.

The final training run is the compute for the one run that produced the released model. The programme is everything that led there: the experiments, the runs that failed, the data collection, the salaries. And serving is the cost of answering every user for the life of the model. The three differ by an order of magnitude or more, and the smallest of them is the one that gets quoted.

Some actual figures

GPT-3, trained in 2020, is the best-documented early case because a team including Google researchers estimated it the following year: about 1,287 megawatt-hours of electricity, which is roughly what a hundred and twenty average American homes use in a year, and around 550 tonnes of carbon dioxide. Contemporary estimates of the rented-compute cost were a few million dollars.

For GPT-4, released in 2023, the only official figure is that its chief executive said it cost “more than a hundred million dollars”. Reporting placed the cluster at around twenty-five thousand of the previous generation of data-centre chips, running for about three months.

Meta published its own numbers for its largest 2024 model: just under thirty-one million chip-hours on sixteen thousand of the current generation. At a rented price of a few dollars an hour, that is on the order of a hundred million dollars for the run.

In December 2024 a Chinese lab reported a number that caused a stock-market reaction: its new model’s final run had used 2.79 million chip-hours, which at two dollars an hour came to about 5.6 million dollars. The paper stated, in the same paragraph, that this excluded “prior research and ablation experiments on architectures, algorithms, or data”. Most of the coverage dropped the sentence. The run was efficient (an order of magnitude cheaper than comparable Western runs) and the number was also the smallest of the three quantities, reported as if it were the largest.

Why the programme costs more

A training run of that size is not attempted once. Before it, dozens of smaller runs test the data mix, the proportions from the scaling laws, the architecture choices. Some full-scale runs fail: the numbers diverge, a bug in the data surfaces at week six, a hardware fault corrupts the checkpoint. Meta’s report on its 2024 run recorded 419 unexpected interruptions over fifty-four days, most from chip failures. Every one of those has a price. A lab that reports its final run alone is telling the truth about a fraction.

Why serving costs most of all

Training happens once. Serving happens every time anyone types. A model used by hundreds of millions of people runs on thousands of chips continuously, and within months of release the electricity and hardware spent answering exceeds what was spent training. Estimates from inside the industry put inference at the majority of total compute spending by 2024, and rising. This is why the previous lesson’s small, over-trained models exist, and why the module on cost per query in this course treats it as the number that matters to you.

The capital above all of it

Underneath the model costs is a building programme. The chips in the cluster cost tens of thousands of dollars each and were bought, not rented, by the largest labs. The data centres that hold them are being announced in tens and hundreds of billions. A frontier model in 2025 is something perhaps a dozen organisations on earth can build, and the reason is not talent; it is that the capital required has passed what a university, a mid-sized company or most governments can commit. The last module of this course returns to what that concentration means.

What this means for you

You do not pay these costs and you do not need to. The cost of a fixed level of capability has been falling by roughly ten times a year since 2021, because a model that matched GPT-3 can now be trained for a small fraction of the price and run on a laptop. A free notebook on Google Colab is enough to adapt a small model to your own data. The frontier is expensive; the year-old frontier is nearly free.

How to read a cost figure

Ask three questions. Is this the final run, the programme, or serving? Is it a published number or a leak? And is the price the market rental rate or the owner’s cost? Labs do not publish their books, and every figure above is an estimate, an official aside or a single paper’s accounting. Treat all of them as order-of-magnitude, and treat a precise number with more suspicion than a rough one.