{"course":{"slug":"choosing-your-tools","title":"Choosing and Using the Tools","url":"https://addaly.com/learn/choosing-your-tools","level":"beginner","track":"work"},"lesson":{"slug":"when-local-is-wrong","title":"When running it yourself is the wrong answer","url":"https://addaly.com/learn/choosing-your-tools/when-local-is-wrong","position":46,"of":76,"minutes":7,"answer":"Local models are the right answer for confidential, offline, high-volume and permanent work, and the wrong one for hard reasoning, whole-product needs, battery life, other people depending on it.","keyIdea":"Local models are the right answer for confidential, offline, high-volume and permanent work, and the wrong one for hard reasoning, whole-product needs, battery life, other people depending on it, and buying hardware to avoid a subscription."},"body":{"format":"markdown","text":"## The enthusiasm and the correction\n\nRunning a model on your own hardware is satisfying and useful, and it is oversold by people who enjoy it. This lesson is the correction, written by somebody who thinks the previous ones in this module are right.\n\n## Seven cases where local is the wrong choice\n\n**You need the hardest reasoning.** A model that fits your laptop is roughly a year behind the frontier on difficult multi-step problems, and the gap is largest on exactly the tasks where you would most value help.\n\n**Your time is worth more than the difference.** Setup is an evening. Keeping up with new releases, formats and runner updates is an hour a month if you care. If a $20 subscription is not a strain, that arithmetic frequently favours paying.\n\n**You are buying hardware to save on subscriptions.** A graphics card costs the equivalent of two to four years of a subscription, will be superseded, and only fits the model sizes it fits. If you want the card anyway, that is a different decision. As a cost-saving measure it usually is not one.\n\n**You need the whole product.** Web search, file handling, memory, a mobile app, voice, sharing with colleagues. You can assemble these around a local model and it is a project, not a setting.\n\n**You are on a laptop battery.** Local inference is a heavy, sustained load. It warms the machine, spins the fans, and shortens the working day considerably.\n\n**You need it to work for other people.** The moment a second person depends on it, you have become the operator of a service: availability, updates, security patches, and someone to blame when it is down.\n\n**You are relying on a safety layer.** A downloaded model arrives with whatever safety training its makers gave it and no product layer around it. If it will be used by children, students or the public, that absence is your responsibility to fill.\n\n## The security surface people forget\n\nA self-hosted front-end reachable from the internet is a service you are running, and it needs the same care as any other. The failure pattern is dull and common: an instance put up quickly for convenience, left unpatched and without a login, holding both an API key and a year of documents.\n\nIf you run one, keep it on your own machine or behind authentication, keep it updated, and do not put anything on a public address without deciding deliberately to run a public service.\n\n## Where local is unambiguously right\n\nTo be balanced, the cases where nothing else will do:\n\n- The material must not leave the building, as a matter of law, contract or professional duty.\n- There is no reliable connection.\n- The volume is enormous and the task is routine.\n- You need the same behaviour in five years with no deprecation notice.\n- You are learning how these systems work, which is a good reason on its own.\n- You cannot pay, and this is the route that costs nothing after the download.\n\n## The maintenance nobody mentions\n\nThere is an ongoing cost that never appears in a setup guide. Runners update and occasionally change behaviour. Model formats evolve, and a file that worked last year may need re-downloading in a new format. Better models come out constantly, so the model you installed is quietly becoming the wrong choice. And an operating system upgrade can break a GPU driver stack in ways that take an evening to unpick.\n\nNone of this is difficult. It is an hour a month if you keep up and a lost weekend if you do not, and it is the honest reason many people who set this up enthusiastically are back on a subscription a year later. Decide whether you want that hour before you begin, rather than discovering it afterwards.\n\n## The summary\n\nA local model is a very good second tool and a demanding first one. The arrangement that works for most people is the one at the end of the previous lesson: a local model for the material that requires it, something hosted for everything else, and no ideological commitment to either.\n\nThe mistake in both directions is the same. Choosing on identity (\"I self-host everything\" or \"I only use the best model\") instead of on what the particular piece of work needs."},"check":{"question":"Someone plans to buy a graphics card to avoid a $20 monthly subscription. What does the arithmetic suggest?","options":[{"text":"The card costs several years of subscription, will be superseded, and only fits certain model sizes.","correct":true},{"text":"It is sound if the card can run a 70B model, since that closes the quality gap with a frontier subscription.","correct":false,"tempting":"Rests on a capability claim that does not hold, as a 70B on consumer hardware still trails the frontier on hard reasoning."},{"text":"It pays for itself within a year, since the card also serves other uses and the subscription is pure cost.","correct":false,"tempting":"Counts other uses as savings against this decision, when they would be reasons to buy the card regardless of the subscription."},{"text":"It depends mainly on electricity prices, which over a few years exceed the purchase price of the card.","correct":false,"tempting":"Elevates running cost above the purchase price, when the card itself is the dominant number in this comparison."}]},"prev":{"slug":"licences-as-a-decision","title":"The licence is a business decision","url":"https://addaly.com/learn/choosing-your-tools/licences-as-a-decision"},"next":{"slug":"the-hybrid-setup","title":"A working setup with two models","url":"https://addaly.com/learn/choosing-your-tools/the-hybrid-setup"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'When running it yourself is the wrong answer', in Choosing and Using the Tools. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/choosing-your-tools/when-local-is-wrong (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"When running it yourself is the wrong answer","courseTitle":"Choosing and Using the Tools","module":{"index":5,"slug":"open-models-and-your-own-hardware","title":"Open models and your own hardware"},"answer":null,"keyIdea":"Local models are the right answer for confidential, offline, high-volume and permanent work, and the wrong one for hard reasoning, whole-product needs, battery life, other people depending on it, and buying hardware to avoid a subscription.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-enthusiasm-and-the-correction","runs":[{"text":"The enthusiasm and the correction"}]},{"t":"p","runs":[{"text":"Running a model on your own hardware is satisfying and useful, and it is oversold by people who enjoy it. This lesson is the correction, written by somebody who thinks the previous ones in this module are right."}]},{"t":"h2","id":"seven-cases-where-local-is-the-wrong-choice","runs":[{"text":"Seven cases where local is the wrong choice"}]},{"t":"p","runs":[{"text":"You need the hardest reasoning.","b":true},{"text":" A model that fits your laptop is roughly a year behind the frontier on difficult multi-step problems, and the gap is largest on exactly the tasks where you would most value help."}]},{"t":"p","runs":[{"text":"Your time is worth more than the difference.","b":true},{"text":" Setup is an evening. Keeping up with new releases, formats and runner updates is an hour a month if you care. If a $20 subscription is not a strain, that arithmetic frequently favours paying."}]},{"t":"p","runs":[{"text":"You are buying hardware to save on subscriptions.","b":true},{"text":" A graphics card costs the equivalent of two to four years of a subscription, will be superseded, and only fits the model sizes it fits. If you want the card anyway, that is a different decision. As a cost-saving measure it usually is not one."}]},{"t":"p","runs":[{"text":"You need the whole product.","b":true},{"text":" Web search, file handling, memory, a mobile app, voice, sharing with colleagues. You can assemble these around a local model and it is a project, not a setting."}]},{"t":"p","runs":[{"text":"You are on a laptop battery.","b":true},{"text":" Local inference is a heavy, sustained load. It warms the machine, spins the fans, and shortens the working day considerably."}]},{"t":"p","runs":[{"text":"You need it to work for other people.","b":true},{"text":" The moment a second person depends on it, you have become the operator of a service: availability, updates, security patches, and someone to blame when it is down."}]},{"t":"p","runs":[{"text":"You are relying on a safety layer.","b":true},{"text":" A downloaded model arrives with whatever safety training its makers gave it and no product layer around it. If it will be used by children, students or the public, that absence is your responsibility to fill."}]},{"t":"h2","id":"the-security-surface-people-forget","runs":[{"text":"The security surface people forget"}]},{"t":"p","runs":[{"text":"A self-hosted front-end reachable from the internet is a service you are running, and it needs the same care as any other. The failure pattern is dull and common: an instance put up quickly for convenience, left unpatched and without a login, holding both an API key and a year of documents."}]},{"t":"p","runs":[{"text":"If you run one, keep it on your own machine or behind authentication, keep it updated, and do not put anything on a public address without deciding deliberately to run a public service."}]},{"t":"h2","id":"where-local-is-unambiguously-right","runs":[{"text":"Where local is unambiguously right"}]},{"t":"p","runs":[{"text":"To be balanced, the cases where nothing else will do:"}]},{"t":"ul","items":[[{"text":"The material must not leave the building, as a matter of law, contract or professional duty."}],[{"text":"There is no reliable connection."}],[{"text":"The volume is enormous and the task is routine."}],[{"text":"You need the same behaviour in five years with no deprecation notice."}],[{"text":"You are learning how these systems work, which is a good reason on its own."}],[{"text":"You cannot pay, and this is the route that costs nothing after the download."}]]},{"t":"h2","id":"the-maintenance-nobody-mentions","runs":[{"text":"The maintenance nobody mentions"}]},{"t":"p","runs":[{"text":"There is an ongoing cost that never appears in a setup guide. Runners update and occasionally change behaviour. Model formats evolve, and a file that worked last year may need re-downloading in a new format. Better models come out constantly, so the model you installed is quietly becoming the wrong choice. And an operating system upgrade can break a GPU driver stack in ways that take an evening to unpick."}]},{"t":"p","runs":[{"text":"None of this is difficult. It is an hour a month if you keep up and a lost weekend if you do not, and it is the honest reason many people who set this up enthusiastically are back on a subscription a year later. Decide whether you want that hour before you begin, rather than discovering it afterwards."}]},{"t":"h2","id":"the-summary","runs":[{"text":"The summary"}]},{"t":"p","runs":[{"text":"A local model is a very good second tool and a demanding first one. The arrangement that works for most people is the one at the end of the previous lesson: a local model for the material that requires it, something hosted for everything else, and no ideological commitment to either."}]},{"t":"p","runs":[{"text":"The mistake in both directions is the same. Choosing on identity (“I self-host everything” or “I only use the best model”) instead of on what the particular piece of work needs."}]}],"check":{"question":[{"text":"Someone plans to buy a graphics card to avoid a $20 monthly subscription. What does the arithmetic suggest?"}],"options":[{"text":[{"text":"The card costs several years of subscription, will be superseded, and only fits certain model sizes."}],"correct":true,"tempting":null},{"text":[{"text":"It is sound if the card can run a 70B model, since that closes the quality gap with a frontier subscription."}],"correct":false,"tempting":[{"text":"Rests on a capability claim that does not hold, as a 70B on consumer hardware still trails the frontier on hard reasoning."}]},{"text":[{"text":"It pays for itself within a year, since the card also serves other uses and the subscription is pure cost."}],"correct":false,"tempting":[{"text":"Counts other uses as savings against this decision, when they would be reasons to buy the card regardless of the subscription."}]},{"text":[{"text":"It depends mainly on electricity prices, which over a few years exceed the purchase price of the card."}],"correct":false,"tempting":[{"text":"Elevates running cost above the purchase price, when the card itself is the dominant number in this comparison."}]}],"order":[1,2,3,0],"why":null},"prev":{"slug":"licences-as-a-decision","title":"The licence is a business decision","minutes":8},"next":{"slug":"the-hybrid-setup","title":"A working setup with two models","minutes":8}}}