{"course":{"slug":"running-models-yourself","title":"Running Models Yourself","url":"https://addaly.com/learn/running-models-yourself","level":"intermediate","track":"deep"},"lesson":{"slug":"renting-by-the-hour","title":"Renting a GPU, and the meter that runs while you think","url":"https://addaly.com/learn/running-models-yourself/renting-by-the-hour","position":69,"of":82,"minutes":9,"answer":"Rented GPUs bill wall-clock time rather than tokens, so the engineering is to keep weights on a persistent volume, run jobs as scripts that terminate themselves.","keyIdea":"Rented GPUs bill wall-clock time rather than tokens, so the engineering is to keep weights on a persistent volume, run jobs as scripts that terminate themselves, and reserve interruptible capacity for work that restarts automatically. And renting gives up the privacy argument unless you trust whoever administers the machine."},"body":{"format":"markdown","text":"## A different unit of account\n\nOwning a card costs money per month regardless of what you do. Renting one costs money per minute the instance exists, not per minute it computes. That single difference reshapes the engineering, because the thing to minimise is wall-clock time with the meter running, and most of that time is usually not generation.\n\nWhere rented hours actually go, in the order people are surprised by:\n\n- **Downloading weights.** A 40GB model at 200 MB/s is about three and a half minutes, and on a slower link it is twenty. You pay for every one of them, on every cold start, unless the weights are on a persistent volume.\n- **Building the environment.** Installing an engine and compiling kernels can take longer than the job.\n- **Thinking.** An instance left running while you read the output, edit a prompt and try again bills continuously.\n- **Forgetting.** The single largest rental bill anyone reports is an instance nobody shut down.\n\n## The discipline that makes it cheap\n\nTreat a rented GPU as a batch machine, even when the work feels interactive. The pattern:\n\n```bash\n#!/usr/bin/env bash\nset -euo pipefail\n# weights on a persistent volume, so this is a no-op after the first run\nexport HF_HOME=/workspace/hf\nhuggingface-cli download \"$MODEL\" --local-dir-use-symlinks False\npython run_job.py --in /workspace/in.jsonl --out /workspace/out.jsonl\naws s3 cp /workspace/out.jsonl \"$RESULTS_URI\"   # or any object store\nsudo shutdown -h now                             # the important line\n```\n\nThe last line is the one people leave out. An instance that terminates itself cannot be forgotten. Develop the job against a small model on your laptop, or against a few rows, and send the full run to the rented machine only when it is known to work.\n\nKeep a persistent volume for the weights and the environment. Storage while the instance is stopped is billed at a small fraction of the compute rate, and it converts a twenty-minute cold start into a one-minute one. Check the egress policy too: pulling results out is usually cheap or free, but some providers charge for it and a large generated dataset is not small.\n\n## Interruptible instances\n\nSpot or interruptible capacity is substantially cheaper than on-demand, and it can be reclaimed with little warning. That is an easy trade for the right shape of work and a bad one otherwise.\n\nSuitable: anything idempotent and chunked. Write results incrementally, record which inputs are done, and make a restart resume rather than repeat. Unsuitable: an interactive service where somebody notices the interruption, or a long single operation with no checkpoint.\n\nThe honest arithmetic is that an interruptible instance at half the price which loses a quarter of its work to interruptions is still cheaper, but only if restarting is automatic. If restarting costs you ten minutes of attention each time, you have traded currency for your own hours, which the next lesson prices.\n\n## Renting versus buying, as one line\n\n```\nmonths to break even = card price / (hourly rate x hours used per month - running cost per month)\n```\n\nWith a card at 1,000 units, a rental at 0.40 per hour, 100 hours of use a month and 10 units a month of electricity, that is 1000 / (40 - 10) = about 33 months. At 400 hours a month it drops to about 6.5 months. So the honest form of the question is not \"is renting cheaper\" but \"how many hours a month will I genuinely use it\", and that number is the one people overestimate.\n\nTwo adjustments make the comparison fair. Subtract the card's expected resale value from its price, which the next lesson covers. And add, to the rental side, the minutes lost to cold starts, because they are billed.\n\n## The free tier\n\nFor learning, several platforms give real GPU time at no cost: hosted notebook services typically offer a limited number of GPU hours a week, with session timeouts and no persistent storage between sessions. They are entirely adequate for working through this course, for quantising a model, for a small fine-tune, and for benchmarking. They are not suitable for serving anything, because sessions end and there is no stable address.\n\nUse them deliberately: mount your own storage for anything you want to keep, and assume the machine disappears when you close the tab.\n\n## What renting costs that owning does not\n\nThe privacy argument for local inference does not survive a rented machine unless you trust the operator. Your weights sit on their disk, your prompts pass through their memory, and the instance is administered by someone else. For open weights this is usually fine. For prompts containing data that legally cannot leave your jurisdiction, renting is the same category of decision as using a hosted API, and should be made with the same care, including checking which country the hardware is physically in, which is not always the country in the provider's address."},"check":{"question":"A team's rented-GPU bill is four times what their token counts suggest it should be. Usage patterns show short interactive sessions spread across the day. What is the most likely cause?","options":[{"text":"Interruptible capacity is being reclaimed and the work repeated, so tokens are generated several times over.","correct":false,"tempting":"Would inflate token counts as well as cost, whereas the described symptom is cost rising while token counts stay low."},{"text":"The provider bills prefill tokens at a higher rate than generated tokens, which short interactive sessions maximise.","correct":false,"tempting":"Imports per-token pricing from hosted APIs; a rented instance bills for time and does not know what a token is."},{"text":"Rental bills the minutes the instance exists, so downloads, setup and the gaps between requests are all paid for.","correct":true},{"text":"The instance type is oversized for the model, so each token costs more than it needs to.","correct":false,"tempting":"An oversized instance raises the hourly rate but not the ratio between billed time and tokens produced, which is what has moved here."}]},"prev":{"slug":"buying-the-card","title":"Buying hardware, in the right order","url":"https://addaly.com/learn/running-models-yourself/buying-the-card"},"next":{"slug":"depreciation-and-the-resale-floor","title":"The card as an asset, and the sunk-cost trap","url":"https://addaly.com/learn/running-models-yourself/depreciation-and-the-resale-floor"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Renting a GPU, and the meter that runs while you think', in Running Models Yourself. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/running-models-yourself/renting-by-the-hour (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Renting a GPU, and the meter that runs while you think","courseTitle":"Running Models Yourself","module":{"index":8,"slug":"what-it-actually-costs","title":"What it actually costs"},"answer":null,"keyIdea":"Rented GPUs bill wall-clock time rather than tokens, so the engineering is to keep weights on a persistent volume, run jobs as scripts that terminate themselves, and reserve interruptible capacity for work that restarts automatically. And renting gives up the privacy argument unless you trust whoever administers the machine.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"a-different-unit-of-account","runs":[{"text":"A different unit of account"}]},{"t":"p","runs":[{"text":"Owning a card costs money per month regardless of what you do. Renting one costs money per minute the instance exists, not per minute it computes. That single difference reshapes the engineering, because the thing to minimise is wall-clock time with the meter running, and most of that time is usually not generation."}]},{"t":"p","runs":[{"text":"Where rented hours actually go, in the order people are surprised by:"}]},{"t":"ul","items":[[{"text":"Downloading weights.","b":true},{"text":" A 40GB model at 200 MB/s is about three and a half minutes, and on a slower link it is twenty. You pay for every one of them, on every cold start, unless the weights are on a persistent volume."}],[{"text":"Building the environment.","b":true},{"text":" Installing an engine and compiling kernels can take longer than the job."}],[{"text":"Thinking.","b":true},{"text":" An instance left running while you read the output, edit a prompt and try again bills continuously."}],[{"text":"Forgetting.","b":true},{"text":" The single largest rental bill anyone reports is an instance nobody shut down."}]]},{"t":"h2","id":"the-discipline-that-makes-it-cheap","runs":[{"text":"The discipline that makes it cheap"}]},{"t":"p","runs":[{"text":"Treat a rented GPU as a batch machine, even when the work feels interactive. The pattern:"}]},{"t":"code","lang":"bash","label":"Bash","text":"#!/usr/bin/env bash\nset -euo pipefail\n# weights on a persistent volume, so this is a no-op after the first run\nexport HF_HOME=/workspace/hf\nhuggingface-cli download \"$MODEL\" --local-dir-use-symlinks False\npython run_job.py --in /workspace/in.jsonl --out /workspace/out.jsonl\naws s3 cp /workspace/out.jsonl \"$RESULTS_URI\"   # or any object store\nsudo shutdown -h now                             # the important line"},{"t":"p","runs":[{"text":"The last line is the one people leave out. An instance that terminates itself cannot be forgotten. Develop the job against a small model on your laptop, or against a few rows, and send the full run to the rented machine only when it is known to work."}]},{"t":"p","runs":[{"text":"Keep a persistent volume for the weights and the environment. Storage while the instance is stopped is billed at a small fraction of the compute rate, and it converts a twenty-minute cold start into a one-minute one. Check the egress policy too: pulling results out is usually cheap or free, but some providers charge for it and a large generated dataset is not small."}]},{"t":"h2","id":"interruptible-instances","runs":[{"text":"Interruptible instances"}]},{"t":"p","runs":[{"text":"Spot or interruptible capacity is substantially cheaper than on-demand, and it can be reclaimed with little warning. That is an easy trade for the right shape of work and a bad one otherwise."}]},{"t":"p","runs":[{"text":"Suitable: anything idempotent and chunked. Write results incrementally, record which inputs are done, and make a restart resume rather than repeat. Unsuitable: an interactive service where somebody notices the interruption, or a long single operation with no checkpoint."}]},{"t":"p","runs":[{"text":"The honest arithmetic is that an interruptible instance at half the price which loses a quarter of its work to interruptions is still cheaper, but only if restarting is automatic. If restarting costs you ten minutes of attention each time, you have traded currency for your own hours, which the next lesson prices."}]},{"t":"h2","id":"renting-versus-buying-as-one-line","runs":[{"text":"Renting versus buying, as one line"}]},{"t":"code","lang":"","label":"Text","text":"months to break even = card price / (hourly rate x hours used per month - running cost per month)"},{"t":"p","runs":[{"text":"With a card at 1,000 units, a rental at 0.40 per hour, 100 hours of use a month and 10 units a month of electricity, that is 1000 / (40 - 10) = about 33 months. At 400 hours a month it drops to about 6.5 months. So the honest form of the question is not “is renting cheaper” but “how many hours a month will I genuinely use it”, and that number is the one people overestimate."}]},{"t":"p","runs":[{"text":"Two adjustments make the comparison fair. Subtract the card’s expected resale value from its price, which the next lesson covers. And add, to the rental side, the minutes lost to cold starts, because they are billed."}]},{"t":"h2","id":"the-free-tier","runs":[{"text":"The free tier"}]},{"t":"p","runs":[{"text":"For learning, several platforms give real GPU time at no cost: hosted notebook services typically offer a limited number of GPU hours a week, with session timeouts and no persistent storage between sessions. They are entirely adequate for working through this course, for quantising a model, for a small fine-tune, and for benchmarking. They are not suitable for serving anything, because sessions end and there is no stable address."}]},{"t":"p","runs":[{"text":"Use them deliberately: mount your own storage for anything you want to keep, and assume the machine disappears when you close the tab."}]},{"t":"h2","id":"what-renting-costs-that-owning-does-not","runs":[{"text":"What renting costs that owning does not"}]},{"t":"p","runs":[{"text":"The privacy argument for local inference does not survive a rented machine unless you trust the operator. Your weights sit on their disk, your prompts pass through their memory, and the instance is administered by someone else. For open weights this is usually fine. For prompts containing data that legally cannot leave your jurisdiction, renting is the same category of decision as using a hosted API, and should be made with the same care, including checking which country the hardware is physically in, which is not always the country in the provider’s address."}]}],"check":{"question":[{"text":"A team’s rented-GPU bill is four times what their token counts suggest it should be. Usage patterns show short interactive sessions spread across the day. What is the most likely cause?"}],"options":[{"text":[{"text":"Interruptible capacity is being reclaimed and the work repeated, so tokens are generated several times over."}],"correct":false,"tempting":[{"text":"Would inflate token counts as well as cost, whereas the described symptom is cost rising while token counts stay low."}]},{"text":[{"text":"The provider bills prefill tokens at a higher rate than generated tokens, which short interactive sessions maximise."}],"correct":false,"tempting":[{"text":"Imports per-token pricing from hosted APIs; a rented instance bills for time and does not know what a token is."}]},{"text":[{"text":"Rental bills the minutes the instance exists, so downloads, setup and the gaps between requests are all paid for."}],"correct":true,"tempting":null},{"text":[{"text":"The instance type is oversized for the model, so each token costs more than it needs to."}],"correct":false,"tempting":[{"text":"An oversized instance raises the hourly rate but not the ratio between billed time and tokens produced, which is what has moved here."}]}],"order":[3,0,1,2],"why":null},"prev":{"slug":"buying-the-card","title":"Buying hardware, in the right order","minutes":9},"next":{"slug":"depreciation-and-the-resale-floor","title":"The card as an asset, and the sunk-cost trap","minutes":8}}}