{"course":{"slug":"open-models","title":"The Open Model Ecosystem","url":"https://addaly.com/learn/open-models","level":"intermediate","track":"deep"},"lesson":{"slug":"rent-or-own","title":"Rent, or own: the arithmetic","url":"https://addaly.com/learn/open-models/rent-or-own","position":39,"of":71,"minutes":9,"answer":"Breakeven against a rented GPU sits in the hundreds of millions of tokens a month and assumes high utilisation, so the real reasons to self-host are usually privacy, offline operation and permanence rather than cost.","keyIdea":"Breakeven against a rented GPU sits in the hundreds of millions of tokens a month and assumes high utilisation, so the real reasons to self-host are usually privacy, offline operation and permanence rather than cost."},"body":{"format":"markdown","text":"## Three ways to buy inference\n\n**Per token**, from an inference provider. You pay for what you use, there is nothing to operate, and the price already includes their batching efficiency.\n\n**Per hour**, renting a GPU from a cloud. You pay whether or not anything is running.\n\n**Outright**, buying hardware. You pay once, plus electricity, plus your time.\n\nThe decision is arithmetic plus three things arithmetic cannot capture. Do the arithmetic first, because it eliminates most cases.\n\n## The numbers, at the time of writing\n\nPer-token prices for open models sit roughly at a few tens of cents per million tokens for an 8B class model and a few dollars per million for a 70B, with output priced two to four times input, and a three to five times spread between providers for the identical model.\n\nAn A100 with 80 GB rents for somewhere between one and two dollars an hour depending on the provider and commitment, which is roughly 700 to 1,500 dollars a month running continuously. A consumer card such as a used 3090 with 24 GB costs a few hundred to about a thousand to buy, and draws 300 to 450 watts under load: at typical electricity prices, ten to thirty dollars a month if it runs all day.\n\n## The breakeven, and why it usually favours renting\n\nTake a 70B model at roughly two dollars per million output tokens rented per token, against an 80 GB GPU at about 1,100 dollars a month. Breakeven is around 550 million output tokens a month.\n\n> Figure: Renting a 70B per token against owning an 80 GB GPU. Line chart of Dollars per month against Million output tokens per month: Rented per token, about $2 per million output (0, 0) (100, 200) (300, 600) (550, 1100) (800, 1600); One 80 GB GPU rented by the hour (0, 1100) (800, 1100). Breakeven is around 550 million output tokens a month, roughly 18 million a day, every day. Most products are two orders of magnitude below it. And the flat line assumes the card is busy: at five per cent utilisation it costs twenty times its sticker price per token delivered.\n\nThat is a very large number. It is roughly 18 million tokens a day, or about 7,000 substantial responses an hour, every hour, all month. Most products are two orders of magnitude below it.\n\nAnd that comparison assumes you keep the GPU **busy**, which is the assumption that fails. A GPU at 5% utilisation costs twenty times its sticker price per token delivered. Real traffic is bursty: quiet at night, peaked in the afternoon. A rented endpoint absorbs that for free because someone else's traffic fills the gaps. Your own machine does not.\n\nSo the honest default is: **rent per token until you have steady, high, predictable load.** The exceptions are not about cost.\n\n## The three reasons that are not arithmetic\n\n**Data cannot leave.** Health records, legal files, recordings of conversations, anything under a data residency obligation or a contract you signed. This is the most common genuine reason to self-host and it does not need a cost justification, though it is worth stating the cost honestly so the decision is made with open eyes.\n\n**It must work offline.** A factory floor, a field device, a clinic with intermittent connectivity, a laptop on a plane. No amount of price advantage helps a service that is unreachable.\n\n**Nobody can take it away.** A rented model can be deprecated, reconfigured, price-changed or restricted. Weights on your disk cannot. For a product whose behaviour must be stable for years, this is a real form of insurance.\n\nThere is a fourth that people undercount: **learning**. Running the stack yourself teaches you what the previous six lessons describe. That has value even when it is not the cheapest option.\n\n## The costs people leave out\n\nWhen someone presents a spreadsheet showing self-hosting is cheaper, check for these:\n\n- **Utilisation.** Divide by the fraction of the month the machine is actually working. This is usually the whole error.\n- **Redundancy.** One GPU is a single point of failure. Two doubles the cost.\n- **The person.** Somebody has to patch, monitor, respond at 3am and handle upgrades. A fraction of an engineer costs more than the hardware.\n- **Model refresh.** A better model arrives every few months. Re-quantising, re-evaluating and re-deploying is recurring work.\n- **Idle time in development.** The machine you bought for production is also the one people leave a notebook running on.\n\n## A decision rule you can use today\n\n1. If data may not leave, self-host and stop reading.\n2. If it must work offline, self-host and stop reading.\n3. Otherwise, estimate monthly tokens honestly, multiply by a provider's price, and compare against a rented GPU at your *realistic* utilisation, not at 100%.\n4. If renting per token is cheaper, rent, and revisit at ten times the volume.\n5. Buy hardware for learning, for a workstation, or for constant load, not to save money on a service that runs for two hours a day."},"check":{"question":"A team argues that renting a GPU at 1,100 dollars a month beats paying 900 dollars a month to a per-token provider. Their traffic runs about six hours a day on weekdays. What is wrong with the comparison?","options":[{"text":"Nothing; the fixed cost is higher but the GPU also removes network latency, which is worth the difference.","correct":false,"tempting":"Latency to a provider is usually tens of milliseconds against generation times measured in seconds, so it rarely justifies a cost inversion."},{"text":"The GPU is idle around 82% of the time, so its effective cost per delivered token is several times what the monthly figure suggests, while the provider bills only for use.","correct":true},{"text":"The provider's price will fall over time, so the comparison should use projected rather than current prices.","correct":false,"tempting":"True as a trend and worth noting, but it is not the error in the arithmetic as presented."},{"text":"A rented GPU cannot serve the same model quality as a provider, since providers use larger hardware.","correct":false,"tempting":"The same weights on adequate hardware give the same quality; the difference is in batching efficiency and utilisation, not capability."}]},"prev":{"slug":"free-compute","title":"Running all of this without money","url":"https://addaly.com/learn/open-models/free-compute"},"next":{"slug":"models-on-phones","title":"What actually runs on a phone","url":"https://addaly.com/learn/open-models/models-on-phones"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Rent, or own: the arithmetic', in The Open Model Ecosystem. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/open-models/rent-or-own (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Rent, or own: the arithmetic","courseTitle":"The Open Model Ecosystem","module":{"index":4,"slug":"making-it-run","title":"Making it run on the machine you have"},"answer":null,"keyIdea":"Breakeven against a rented GPU sits in the hundreds of millions of tokens a month and assumes high utilisation, so the real reasons to self-host are usually privacy, offline operation and permanence rather than cost.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"line","title":"Renting a 70B per token against owning an 80 GB GPU","series":[{"label":"Rented per token, about $2 per million output","points":[[0,0],[100,200],[300,600],[550,1100],[800,1600]]},{"label":"One 80 GB GPU rented by the hour","points":[[0,1100],[800,1100]]}],"xLabel":"Million output tokens per month","yLabel":"Dollars per month","caption":"Breakeven is around 550 million output tokens a month, roughly 18 million a day, every day. Most products are two orders of magnitude below it. And the flat line assumes the card is busy: at five per cent utilisation it costs twenty times its sticker price per token delivered."},"alt":"Figure: Renting a 70B per token against owning an 80 GB GPU. Line chart of Dollars per month against Million output tokens per month: Rented per token, about $2 per million output (0, 0) (100, 200) (300, 600) (550, 1100) (800, 1600); One 80 GB GPU rented by the hour (0, 1100) (800, 1100). Breakeven is around 550 million output tokens a month, roughly 18 million a day, every day. Most products are two orders of magnitude below it. And the flat line assumes the card is busy: at five per cent utilisation it costs twenty times its sticker price per token delivered."}},"blocks":[{"t":"h2","id":"three-ways-to-buy-inference","runs":[{"text":"Three ways to buy inference"}]},{"t":"p","runs":[{"text":"Per token","b":true},{"text":", from an inference provider. You pay for what you use, there is nothing to operate, and the price already includes their batching efficiency."}]},{"t":"p","runs":[{"text":"Per hour","b":true},{"text":", renting a GPU from a cloud. You pay whether or not anything is running."}]},{"t":"p","runs":[{"text":"Outright","b":true},{"text":", buying hardware. You pay once, plus electricity, plus your time."}]},{"t":"p","runs":[{"text":"The decision is arithmetic plus three things arithmetic cannot capture. Do the arithmetic first, because it eliminates most cases."}]},{"t":"h2","id":"the-numbers-at-the-time-of-writing","runs":[{"text":"The numbers, at the time of writing"}]},{"t":"p","runs":[{"text":"Per-token prices for open models sit roughly at a few tens of cents per million tokens for an 8B class model and a few dollars per million for a 70B, with output priced two to four times input, and a three to five times spread between providers for the identical model."}]},{"t":"p","runs":[{"text":"An A100 with 80 GB rents for somewhere between one and two dollars an hour depending on the provider and commitment, which is roughly 700 to 1,500 dollars a month running continuously. A consumer card such as a used 3090 with 24 GB costs a few hundred to about a thousand to buy, and draws 300 to 450 watts under load: at typical electricity prices, ten to thirty dollars a month if it runs all day."}]},{"t":"h2","id":"the-breakeven-and-why-it-usually-favours-renting","runs":[{"text":"The breakeven, and why it usually favours renting"}]},{"t":"p","runs":[{"text":"Take a 70B model at roughly two dollars per million output tokens rented per token, against an 80 GB GPU at about 1,100 dollars a month. Breakeven is around 550 million output tokens a month."}]},{"t":"p","runs":[{"text":"That is a very large number. It is roughly 18 million tokens a day, or about 7,000 substantial responses an hour, every hour, all month. Most products are two orders of magnitude below it."}]},{"t":"p","runs":[{"text":"And that comparison assumes you keep the GPU "},{"text":"busy","b":true},{"text":", which is the assumption that fails. A GPU at 5% utilisation costs twenty times its sticker price per token delivered. Real traffic is bursty: quiet at night, peaked in the afternoon. A rented endpoint absorbs that for free because someone else’s traffic fills the gaps. Your own machine does not."}]},{"t":"p","runs":[{"text":"So the honest default is: "},{"text":"rent per token until you have steady, high, predictable load.","b":true},{"text":" The exceptions are not about cost."}]},{"t":"h2","id":"the-three-reasons-that-are-not-arithmetic","runs":[{"text":"The three reasons that are not arithmetic"}]},{"t":"p","runs":[{"text":"Data cannot leave.","b":true},{"text":" Health records, legal files, recordings of conversations, anything under a data residency obligation or a contract you signed. This is the most common genuine reason to self-host and it does not need a cost justification, though it is worth stating the cost honestly so the decision is made with open eyes."}]},{"t":"p","runs":[{"text":"It must work offline.","b":true},{"text":" A factory floor, a field device, a clinic with intermittent connectivity, a laptop on a plane. No amount of price advantage helps a service that is unreachable."}]},{"t":"p","runs":[{"text":"Nobody can take it away.","b":true},{"text":" A rented model can be deprecated, reconfigured, price-changed or restricted. Weights on your disk cannot. For a product whose behaviour must be stable for years, this is a real form of insurance."}]},{"t":"p","runs":[{"text":"There is a fourth that people undercount: "},{"text":"learning","b":true},{"text":". Running the stack yourself teaches you what the previous six lessons describe. That has value even when it is not the cheapest option."}]},{"t":"h2","id":"the-costs-people-leave-out","runs":[{"text":"The costs people leave out"}]},{"t":"p","runs":[{"text":"When someone presents a spreadsheet showing self-hosting is cheaper, check for these:"}]},{"t":"ul","items":[[{"text":"Utilisation.","b":true},{"text":" Divide by the fraction of the month the machine is actually working. This is usually the whole error."}],[{"text":"Redundancy.","b":true},{"text":" One GPU is a single point of failure. Two doubles the cost."}],[{"text":"The person.","b":true},{"text":" Somebody has to patch, monitor, respond at 3am and handle upgrades. A fraction of an engineer costs more than the hardware."}],[{"text":"Model refresh.","b":true},{"text":" A better model arrives every few months. Re-quantising, re-evaluating and re-deploying is recurring work."}],[{"text":"Idle time in development.","b":true},{"text":" The machine you bought for production is also the one people leave a notebook running on."}]]},{"t":"h2","id":"a-decision-rule-you-can-use-today","runs":[{"text":"A decision rule you can use today"}]},{"t":"ol","start":1,"items":[[{"text":"If data may not leave, self-host and stop reading."}],[{"text":"If it must work offline, self-host and stop reading."}],[{"text":"Otherwise, estimate monthly tokens honestly, multiply by a provider’s price, and compare against a rented GPU at your "},{"text":"realistic","i":true},{"text":" utilisation, not at 100%."}],[{"text":"If renting per token is cheaper, rent, and revisit at ten times the volume."}],[{"text":"Buy hardware for learning, for a workstation, or for constant load, not to save money on a service that runs for two hours a day."}]]}],"check":{"question":[{"text":"A team argues that renting a GPU at 1,100 dollars a month beats paying 900 dollars a month to a per-token provider. Their traffic runs about six hours a day on weekdays. What is wrong with the comparison?"}],"options":[{"text":[{"text":"Nothing; the fixed cost is higher but the GPU also removes network latency, which is worth the difference."}],"correct":false,"tempting":[{"text":"Latency to a provider is usually tens of milliseconds against generation times measured in seconds, so it rarely justifies a cost inversion."}]},{"text":[{"text":"The GPU is idle around 82% of the time, so its effective cost per delivered token is several times what the monthly figure suggests, while the provider bills only for use."}],"correct":true,"tempting":null},{"text":[{"text":"The provider’s price will fall over time, so the comparison should use projected rather than current prices."}],"correct":false,"tempting":[{"text":"True as a trend and worth noting, but it is not the error in the arithmetic as presented."}]},{"text":[{"text":"A rented GPU cannot serve the same model quality as a provider, since providers use larger hardware."}],"correct":false,"tempting":[{"text":"The same weights on adequate hardware give the same quality; the difference is in batching efficiency and utilisation, not capability."}]}],"order":[2,0,3,1],"why":null},"prev":{"slug":"free-compute","title":"Running all of this without money","minutes":8},"next":{"slug":"models-on-phones","title":"What actually runs on a phone","minutes":8}}}