{"course":{"slug":"running-models-yourself","title":"Running Models Yourself","url":"https://addaly.com/learn/running-models-yourself","level":"intermediate","track":"deep"},"lesson":{"slug":"the-honest-cost-comparison","title":"The honest cost comparison","url":"https://addaly.com/learn/running-models-yourself/the-honest-cost-comparison","position":65,"of":82,"minutes":10,"answer":"Local cost per token is set by how busy the GPU is; an idle card generates the most expensive tokens there are.","keyIdea":"Local cost per token is set by how busy the GPU is; an idle card generates the most expensive tokens there are."},"body":{"format":"markdown","text":"\"Local is free\" is the most common false claim in this field, and \"local is never cheaper\" is the second. Both dissolve under arithmetic. Here it is, with numbers you can substitute your own into.\n\n## Electricity per million tokens\n\nMeasure at the wall if you have a meter. Otherwise: GPU power draw plus 80 to 120 W for the rest of the machine.\n\nTake a used 24 GB card at 350 W under load, plus 100 W system, so 0.45 kW. At $0.15/kWh that is **$0.0675 per busy hour**.\n\nNow convert to tokens.\n\n**One user, 8B at Q4, 60 tok/s:** 216,000 tokens/hour.\n\n> Figure: One card, and a hundredfold spread in cost per token. Bars (US cents per million output tokens): Batched at 32, electricity only 1.2; Batched at 32, card included, busy all day 1.7; One user at 60 tok/s, electricity only 31; One user two hours a day, card included 179. A 24 GB card drawing 350 W plus 100 W of system, at fifteen cents a kilowatt-hour, bought for 700 and held three years. Nothing about the model, the card or the electricity changes between the first row and the last. Only how much of the time the card was doing work.\n\n```\n$0.0675 / 0.216M tokens = $0.31 per million output tokens\n```\n\n**Batched, 32 concurrent requests, ~1,600 tok/s aggregate:** 5,760,000 tokens/hour.\n\n```\n$0.0675 / 5.76M tokens = $0.012 per million output tokens\n```\n\nSame hardware, same electricity bill, **26 times cheaper per token**. That single comparison is the whole economics of self-hosting: local is cheap when the GPU is busy, and expensive when it is not.\n\n## Hardware amortised\n\nSay the card cost $700 and lasts three years.\n\n- **Busy 24/7:** 26,280 hours, so $0.027/hour. Adds about $0.12 per million at single-stream, half a cent at batch 32.\n- **Two hours a day:** 2,190 hours, so $0.32/hour. Adds about **$1.48 per million** at single-stream.\n\nAn idle GPU you have already bought produces the most expensive tokens you will ever generate.\n\n## Against API prices\n\nCheck current prices, because they move, but the shape holds. Hosted 8B-class open models sit around $0.05 to $0.60 per million output tokens. Frontier models sit around $1.50 to $15.\n\n**Break-even in tokens:**\n\n```\nhardware_cost / (api_price_per_token - your_electricity_per_token)\n```\n\nAt a generous $0.60/M API price and $0.012/M electricity:\n\n```\n$700 / $0.588 per M = 1.19 billion output tokens\n```\n\nAt batch throughput of 5.76M tokens/hour, that is **207 hours, about nine days of a saturated GPU.** At two hours a day of one person chatting at 60 tok/s, the same 1.19 billion tokens takes roughly **seven and a half years.**\n\nIf the API alternative is a cheaper $0.10/M model, break-even moves to about 8 billion tokens: still under two months of a busy card, still never for casual chat.\n\n## Where you are changes the answer\n\nElectricity is roughly $0.10/kWh across much of the United States and India, $0.25 in the UK, $0.30 to 0.40 in Germany and Denmark, and under $0.05 in parts of the Gulf. At $0.35/kWh the single-stream figure becomes $0.73 per million, and local loses outright on cost at low utilisation. Substitute your own rate; the sums above take thirty seconds to redo.\n\n## The costs people leave out\n\nYour time, which usually exceeds everything else combined for light use. The machine you cannot use for anything else while a model is resident. Cooling, and noise if it sits in a room you work in. And the second card you buy eight months later.\n\n## The three honest verdicts\n\n**One person, chatting a few hours a day, small model.** The API is cheaper, often by ten times or more. Run locally for privacy, offline capability, or control, and say that plainly instead of calling it a saving.\n\n**Sustained batch work** (classifying five million documents, bulk rewriting, generating embeddings, an overnight pipeline) **on hardware you own.** Local can be five to fifty times cheaper, and the batching arithmetic above is exactly why.\n\n**Frontier-quality output.** There is no local option at a price you would pay. A machine that runs the largest open models well costs more than a decade of most people's API bills, and depreciates while it sits there.\n\nWork out your own break-even before you buy anything. It is one division."},"check":{"question":"Two teams each buy the same $700 GPU to replace the same API. Team A runs a nightly classification job that saturates the card for six hours. Team B has five people chatting through the day, keeping the GPU busy about forty minutes in total. Who saves money?","options":[{"text":"Team A, because cost per token is set by utilisation: batched hours produce far more tokens per kWh and spread the hardware cost over vastly more output.","correct":true},{"text":"Both equally, since the hardware cost is fixed and each team's API bill drops to zero.","correct":false,"tempting":"Forgets the API bill was proportional to usage, so Team B was barely paying anything to begin with."},{"text":"Team B, because interactive use avoids the per-request overhead that API pricing builds in.","correct":false,"tempting":"Invents a pricing structure; hosted models bill per token, so light interactive use is exactly where they are cheapest."},{"text":"Neither, because electricity per token always works out higher than published API prices.","correct":false,"tempting":"Takes the single-stream electricity figure as universal and ignores that batching cuts it by more than twenty times."}]},"prev":{"slug":"hybrid-local-and-api","title":"Local first, API behind it","url":"https://addaly.com/learn/running-models-yourself/hybrid-local-and-api"},"next":{"slug":"utilisation-decides-everything","title":"Duty cycle, and why an idle card is expensive","url":"https://addaly.com/learn/running-models-yourself/utilisation-decides-everything"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The honest cost comparison', in Running Models Yourself. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/running-models-yourself/the-honest-cost-comparison (Accessed: 31 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The honest cost comparison","courseTitle":"Running Models Yourself","module":{"index":8,"slug":"what-it-actually-costs","title":"What it actually costs"},"answer":null,"keyIdea":"Local cost per token is set by how busy the GPU is; an idle card generates the most expensive tokens there are.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure 8.1","figure":{"type":"bars","unit":"US cents per million output tokens","title":"One card, and a hundredfold spread in cost per token","series":[{"label":"Batched at 32, electricity only","value":1.2},{"label":"Batched at 32, card included, busy all day","value":1.7,"highlight":true},{"label":"One user at 60 tok/s, electricity only","value":31},{"label":"One user two hours a day, card included","value":179}],"caption":"A 24 GB card drawing 350 W plus 100 W of system, at fifteen cents a kilowatt-hour, bought for 700 and held three years. Nothing about the model, the card or the electricity changes between the first row and the last. Only how much of the time the card was doing work."},"alt":"Figure: One card, and a hundredfold spread in cost per token. Bars (US cents per million output tokens): Batched at 32, electricity only 1.2; Batched at 32, card included, busy all day 1.7; One user at 60 tok/s, electricity only 31; One user two hours a day, card included 179. A 24 GB card drawing 350 W plus 100 W of system, at fifteen cents a kilowatt-hour, bought for 700 and held three years. Nothing about the model, the card or the electricity changes between the first row and the last. Only how much of the time the card was doing work."}},"blocks":[{"t":"p","runs":[{"text":"“Local is free” is the most common false claim in this field, and “local is never cheaper” is the second. Both dissolve under arithmetic. Here it is, with numbers you can substitute your own into."}],"lede":true},{"t":"h2","id":"electricity-per-million-tokens","runs":[{"text":"Electricity per million tokens"}]},{"t":"p","runs":[{"text":"Measure at the wall if you have a meter. Otherwise: GPU power draw plus 80 to 120 W for the rest of the machine."}]},{"t":"p","runs":[{"text":"Take a used 24 GB card at 350 W under load, plus 100 W system, so 0.45 kW. At $0.15/kWh that is "},{"text":"$0.0675 per busy hour","b":true},{"text":"."}]},{"t":"p","runs":[{"text":"Now convert to tokens."}]},{"t":"p","runs":[{"text":"One user, 8B at Q4, 60 tok/s:","b":true},{"text":" 216,000 tokens/hour."}]},{"t":"code","lang":"","label":"Text","text":"$0.0675 / 0.216M tokens = $0.31 per million output tokens"},{"t":"p","runs":[{"text":"Batched, 32 concurrent requests, ~1,600 tok/s aggregate:","b":true},{"text":" 5,760,000 tokens/hour."}]},{"t":"code","lang":"","label":"Text","text":"$0.0675 / 5.76M tokens = $0.012 per million output tokens"},{"t":"p","runs":[{"text":"Same hardware, same electricity bill, "},{"text":"26 times cheaper per token","b":true},{"text":". That single comparison is the whole economics of self-hosting: local is cheap when the GPU is busy, and expensive when it is not."}]},{"t":"h2","id":"hardware-amortised","runs":[{"text":"Hardware amortised"}]},{"t":"p","runs":[{"text":"Say the card cost $700 and lasts three years."}]},{"t":"ul","items":[[{"text":"Busy 24/7:","b":true},{"text":" 26,280 hours, so $0.027/hour. Adds about $0.12 per million at single-stream, half a cent at batch 32."}],[{"text":"Two hours a day:","b":true},{"text":" 2,190 hours, so $0.32/hour. Adds about "},{"text":"$1.48 per million","b":true},{"text":" at single-stream."}]]},{"t":"p","runs":[{"text":"An idle GPU you have already bought produces the most expensive tokens you will ever generate."}]},{"t":"h2","id":"against-api-prices","runs":[{"text":"Against API prices"}]},{"t":"p","runs":[{"text":"Check current prices, because they move, but the shape holds. Hosted 8B-class open models sit around $0.05 to $0.60 per million output tokens. Frontier models sit around $1.50 to $15."}]},{"t":"p","runs":[{"text":"Break-even in tokens:","b":true}]},{"t":"code","lang":"","label":"Text","text":"hardware_cost / (api_price_per_token - your_electricity_per_token)"},{"t":"p","runs":[{"text":"At a generous $0.60/M API price and $0.012/M electricity:"}]},{"t":"code","lang":"","label":"Text","text":"$700 / $0.588 per M = 1.19 billion output tokens"},{"t":"p","runs":[{"text":"At batch throughput of 5.76M tokens/hour, that is "},{"text":"207 hours, about nine days of a saturated GPU.","b":true},{"text":" At two hours a day of one person chatting at 60 tok/s, the same 1.19 billion tokens takes roughly "},{"text":"seven and a half years.","b":true}]},{"t":"p","runs":[{"text":"If the API alternative is a cheaper $0.10/M model, break-even moves to about 8 billion tokens: still under two months of a busy card, still never for casual chat."}]},{"t":"h2","id":"where-you-are-changes-the-answer","runs":[{"text":"Where you are changes the answer"}]},{"t":"p","runs":[{"text":"Electricity is roughly $0.10/kWh across much of the United States and India, $0.25 in the UK, $0.30 to 0.40 in Germany and Denmark, and under $0.05 in parts of the Gulf. At $0.35/kWh the single-stream figure becomes $0.73 per million, and local loses outright on cost at low utilisation. Substitute your own rate; the sums above take thirty seconds to redo."}]},{"t":"h2","id":"the-costs-people-leave-out","runs":[{"text":"The costs people leave out"}]},{"t":"p","runs":[{"text":"Your time, which usually exceeds everything else combined for light use. The machine you cannot use for anything else while a model is resident. Cooling, and noise if it sits in a room you work in. And the second card you buy eight months later."}]},{"t":"h2","id":"the-three-honest-verdicts","runs":[{"text":"The three honest verdicts"}]},{"t":"p","runs":[{"text":"One person, chatting a few hours a day, small model.","b":true},{"text":" The API is cheaper, often by ten times or more. Run locally for privacy, offline capability, or control, and say that plainly instead of calling it a saving."}]},{"t":"p","runs":[{"text":"Sustained batch work","b":true},{"text":" (classifying five million documents, bulk rewriting, generating embeddings, an overnight pipeline) "},{"text":"on hardware you own.","b":true},{"text":" Local can be five to fifty times cheaper, and the batching arithmetic above is exactly why."}]},{"t":"p","runs":[{"text":"Frontier-quality output.","b":true},{"text":" There is no local option at a price you would pay. A machine that runs the largest open models well costs more than a decade of most people’s API bills, and depreciates while it sits there."}]},{"t":"p","runs":[{"text":"Work out your own break-even before you buy anything. It is one division."}]}],"check":{"question":[{"text":"Two teams each buy the same $700 GPU to replace the same API. Team A runs a nightly classification job that saturates the card for six hours. Team B has five people chatting through the day, keeping the GPU busy about forty minutes in total. Who saves money?"}],"options":[{"text":[{"text":"Team A, because cost per token is set by utilisation: batched hours produce far more tokens per kWh and spread the hardware cost over vastly more output."}],"correct":true,"tempting":null},{"text":[{"text":"Both equally, since the hardware cost is fixed and each team’s API bill drops to zero."}],"correct":false,"tempting":[{"text":"Forgets the API bill was proportional to usage, so Team B was barely paying anything to begin with."}]},{"text":[{"text":"Team B, because interactive use avoids the per-request overhead that API pricing builds in."}],"correct":false,"tempting":[{"text":"Invents a pricing structure; hosted models bill per token, so light interactive use is exactly where they are cheapest."}]},{"text":[{"text":"Neither, because electricity per token always works out higher than published API prices."}],"correct":false,"tempting":[{"text":"Takes the single-stream electricity figure as universal and ignores that batching cuts it by more than twenty times."}]}],"order":[1,3,0,2],"why":null},"prev":{"slug":"hybrid-local-and-api","title":"Local first, API behind it","minutes":8},"next":{"slug":"utilisation-decides-everything","title":"Duty cycle, and why an idle card is expensive","minutes":9}}}