{"course":{"slug":"hugging-face","title":"Hugging Face, End to End","url":"https://addaly.com/learn/hugging-face","level":"intermediate","track":"create"},"lesson":{"slug":"running-it-locally","title":"Your disk fills up before your patience does","url":"https://addaly.com/learn/hugging-face/running-it-locally","position":19,"of":75,"minutes":11,"answer":"Weights cost parameters times bytes-per-parameter and everything downloaded lands in one shared cache, where deleting the readable filenames frees nothing because they are only links to the blobs.","keyIdea":"Weights cost parameters times bytes-per-parameter and everything downloaded lands in one shared cache, where deleting the readable filenames frees nothing because they are only links to the blobs."},"body":{"format":"markdown","text":"## Two lines that quietly download two gigabytes\n\nRunning a model yourself is less dramatic than it sounds. Install one package and call one function:\n\n```bash\npip install -U \"transformers[torch]\"\n```\n\n```python\nfrom transformers import pipeline\n\nclf = pipeline(\n    \"text-classification\",\n    model=\"distilbert-base-uncased-finetuned-sst-2-english\",\n)\nprint(clf(\"The bus was late but the driver waited for me.\"))\n```\n\nThat model is around 250 MB and classifies in a fraction of a second on any laptop CPU. No GPU, no configuration. The point of the example is not sentiment analysis, which is a solved and slightly boring task: it is that `pipeline` downloads the weights, the tokenizer and the config, caches them, and gives you a working function. Swap the model name and the same three lines do translation, transcription, captioning or entity extraction.\n\n## The cache, and the symlink that fools you\n\nEverything you download goes to one shared cache, by default `~/.cache/huggingface/hub`. Point it somewhere with room by setting one variable before you start:\n\n```bash\nexport HF_HOME=/mnt/bigdisk/huggingface\n```\n\nInside the cache, each repository has a `blobs/` directory holding the actual bytes and a `snapshots/<commit-hash>/` directory holding symbolic links with the human-readable filenames. Two revisions of the same model share every blob that did not change.\n\nThis structure produces one specific and very common disaster. You run out of disk, you go into the cache, you find a folder full of familiar filenames under `snapshots/`, you delete them, and nothing is freed. You deleted links. The blobs they pointed at are still there. Use the tool instead:\n\n```bash\nhf cache scan\nhf cache delete\n```\n\nOn older versions these are `huggingface-cli scan-cache` and `huggingface-cli delete-cache`. The scan prints every repository, its size and its revisions, which is also how you discover you have four copies of the same 13 GB model from four different tutorials.\n\n## The memory arithmetic\n\nThis is the whole of capacity planning, and it fits in five lines.\n\n- 32-bit weights: 4 bytes per parameter.\n- 16-bit: 2 bytes.\n- 8-bit: 1 byte.\n- 4-bit: roughly half a byte.\n\nSo a 7-billion-parameter model needs about 28, 14, 7 or 4 GB for the weights alone. Then add the key-value cache, which grows with how much context you feed it, and a gigabyte or two of framework overhead. A rule that holds: the weights plus about a quarter again.\n\nThe failure mode matters as much as the number. If a model does not fit in RAM, your machine does not usually raise a clean error: it starts swapping to disk, and the process appears to hang. People wait forty minutes for a first token believing it is slow when it is actually stuck. Check the size before you start, not while you wait.\n\n## What a machine with no GPU actually does\n\nA great deal, once you stop trying to run the wrong things.\n\n**Small models run fine.** Classification, embeddings, named entities, translation with a compact model, Whisper at tiny or base size. These are CPU-friendly because they are small, not because of any special trick.\n\n**Image generation is possible at low step counts.** Most diffusion models need twenty to fifty passes through the network to make one image, which is what makes them hopeless on CPU. Distilled models cut that to one to four:\n\n```bash\npip install -U diffusers torch pillow\n```\n\n```python\nfrom diffusers import AutoPipelineForText2Image\nimport torch\n\npipe = AutoPipelineForText2Image.from_pretrained(\n    \"stabilityai/sd-turbo\", torch_dtype=torch.float32\n).to(\"cpu\")\n\nimg = pipe(\n    \"a steel tumbler of chai on a wooden table, morning light\",\n    num_inference_steps=2,\n    guidance_scale=0.0,\n).images[0]\nimg.save(\"chai.png\")\n```\n\nAbout 2.5 GB to download, tens of seconds per image on a normal CPU, and the quality is well below SDXL. Note the licence: `sd-turbo` is non-commercial, exactly as lesson two warned. If the output is going anywhere near a paying client, this is not the model. [/learn/ai-images](/learn/ai-images) covers the craft side.\n\n**Chat models run through llama.cpp, not through `transformers`.** The GGUF format packs quantised weights into a single file, and the Hub is full of them from community quantisers. `Q4_K_M` is the usual balance of size against damage. Ollama and LM Studio both pull GGUF files straight from the Hub, and LM Studio has a graphical interface on Windows, macOS and Linux, so this path needs no Python at all. Rough expectations: with 8 GB of RAM a 3B model at 4-bit reads at a comfortable speed and a 7B is slow but usable; with 16 GB, 7B to 8B is comfortable. Apple silicon does unusually well because the CPU and GPU share memory. [/learn/running-models-yourself](/learn/running-models-yourself) goes into this properly.\n\n**A phone does not run these.** What a phone runs is a browser pointed at a Space, or very small models compiled for the browser through transformers.js. That is not a limitation to work around; it is the reason lessons four and five exist.\n\n## Do this now\n\nRun `hf cache scan` on whatever machine you have been experimenting on. Most people find several gigabytes they did not know about, and at least one model downloaded twice."},"check":{"question":"A laptop is out of disk. The user opens the Hugging Face cache, finds a `snapshots` folder full of files like `model.safetensors`, deletes several gigabytes of them, and the free space barely changes. Why?","options":[{"text":"The deleted files went to the operating system's trash and still occupy the disk until it is emptied.","correct":false,"tempting":"Reaches for a general desktop behaviour instead of the cache's own layout."},{"text":"The library re-downloaded the models automatically as soon as it noticed they were missing.","correct":false,"tempting":"Assumes an active repair process rather than a passive cache that only fetches when asked."},{"text":"The files under `snapshots` are symbolic links to blobs stored elsewhere in the cache, so removing them deletes the names and not the bytes.","correct":true},{"text":"Those were quantised copies that share weights with the full-precision originals, so the real space is held by the originals.","correct":false,"tempting":"Invents weight sharing between separate model files instead of the deduplication the cache actually does at the blob level."}]},"prev":{"slug":"loads-but-outputs-rubbish","title":"It loaded, it ran, the output is nonsense","url":"https://addaly.com/learn/hugging-face/loads-but-outputs-rubbish"},"next":{"slug":"dtype-and-the-memory-bill","title":"The memory arithmetic you can do in your head","url":"https://addaly.com/learn/hugging-face/dtype-and-the-memory-bill"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Your disk fills up before your patience does', in Hugging Face, End to End. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/hugging-face/running-it-locally (Accessed: 31 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Your disk fills up before your patience does","courseTitle":"Hugging Face, End to End","module":{"index":3,"slug":"running-it-on-what-you-have","title":"Running it on the hardware you actually have"},"answer":null,"keyIdea":"Weights cost parameters times bytes-per-parameter and everything downloaded lands in one shared cache, where deleting the readable filenames frees nothing because they are only links to the blobs.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"two-lines-that-quietly-download-two-gigabytes","runs":[{"text":"Two lines that quietly download two gigabytes"}]},{"t":"p","runs":[{"text":"Running a model yourself is less dramatic than it sounds. Install one package and call one function:"}]},{"t":"code","lang":"bash","label":"Bash","text":"pip install -U \"transformers[torch]\""},{"t":"code","lang":"python","label":"Python","text":"from transformers import pipeline\n\nclf = pipeline(\n    \"text-classification\",\n    model=\"distilbert-base-uncased-finetuned-sst-2-english\",\n)\nprint(clf(\"The bus was late but the driver waited for me.\"))"},{"t":"p","runs":[{"text":"That model is around 250 MB and classifies in a fraction of a second on any laptop CPU. No GPU, no configuration. The point of the example is not sentiment analysis, which is a solved and slightly boring task: it is that "},{"text":"pipeline","code":true},{"text":" downloads the weights, the tokenizer and the config, caches them, and gives you a working function. Swap the model name and the same three lines do translation, transcription, captioning or entity extraction."}]},{"t":"h2","id":"the-cache-and-the-symlink-that-fools-you","runs":[{"text":"The cache, and the symlink that fools you"}]},{"t":"p","runs":[{"text":"Everything you download goes to one shared cache, by default "},{"text":"~/.cache/huggingface/hub","code":true},{"text":". Point it somewhere with room by setting one variable before you start:"}]},{"t":"code","lang":"bash","label":"Bash","text":"export HF_HOME=/mnt/bigdisk/huggingface"},{"t":"p","runs":[{"text":"Inside the cache, each repository has a "},{"text":"blobs/","code":true},{"text":" directory holding the actual bytes and a "},{"text":"snapshots/<commit-hash>/","code":true},{"text":" directory holding symbolic links with the human-readable filenames. Two revisions of the same model share every blob that did not change."}]},{"t":"p","runs":[{"text":"This structure produces one specific and very common disaster. You run out of disk, you go into the cache, you find a folder full of familiar filenames under "},{"text":"snapshots/","code":true},{"text":", you delete them, and nothing is freed. You deleted links. The blobs they pointed at are still there. Use the tool instead:"}]},{"t":"code","lang":"bash","label":"Bash","text":"hf cache scan\nhf cache delete"},{"t":"p","runs":[{"text":"On older versions these are "},{"text":"huggingface-cli scan-cache","code":true},{"text":" and "},{"text":"huggingface-cli delete-cache","code":true},{"text":". The scan prints every repository, its size and its revisions, which is also how you discover you have four copies of the same 13 GB model from four different tutorials."}]},{"t":"h2","id":"the-memory-arithmetic","runs":[{"text":"The memory arithmetic"}]},{"t":"p","runs":[{"text":"This is the whole of capacity planning, and it fits in five lines."}]},{"t":"ul","items":[[{"text":"32-bit weights: 4 bytes per parameter."}],[{"text":"16-bit: 2 bytes."}],[{"text":"8-bit: 1 byte."}],[{"text":"4-bit: roughly half a byte."}]]},{"t":"p","runs":[{"text":"So a 7-billion-parameter model needs about 28, 14, 7 or 4 GB for the weights alone. Then add the key-value cache, which grows with how much context you feed it, and a gigabyte or two of framework overhead. A rule that holds: the weights plus about a quarter again."}]},{"t":"p","runs":[{"text":"The failure mode matters as much as the number. If a model does not fit in RAM, your machine does not usually raise a clean error: it starts swapping to disk, and the process appears to hang. People wait forty minutes for a first token believing it is slow when it is actually stuck. Check the size before you start, not while you wait."}]},{"t":"h2","id":"what-a-machine-with-no-gpu-actually-does","runs":[{"text":"What a machine with no GPU actually does"}]},{"t":"p","runs":[{"text":"A great deal, once you stop trying to run the wrong things."}]},{"t":"p","runs":[{"text":"Small models run fine.","b":true},{"text":" Classification, embeddings, named entities, translation with a compact model, Whisper at tiny or base size. These are CPU-friendly because they are small, not because of any special trick."}]},{"t":"p","runs":[{"text":"Image generation is possible at low step counts.","b":true},{"text":" Most diffusion models need twenty to fifty passes through the network to make one image, which is what makes them hopeless on CPU. Distilled models cut that to one to four:"}]},{"t":"code","lang":"bash","label":"Bash","text":"pip install -U diffusers torch pillow"},{"t":"code","lang":"python","label":"Python","text":"from diffusers import AutoPipelineForText2Image\nimport torch\n\npipe = AutoPipelineForText2Image.from_pretrained(\n    \"stabilityai/sd-turbo\", torch_dtype=torch.float32\n).to(\"cpu\")\n\nimg = pipe(\n    \"a steel tumbler of chai on a wooden table, morning light\",\n    num_inference_steps=2,\n    guidance_scale=0.0,\n).images[0]\nimg.save(\"chai.png\")"},{"t":"p","runs":[{"text":"About 2.5 GB to download, tens of seconds per image on a normal CPU, and the quality is well below SDXL. Note the licence: "},{"text":"sd-turbo","code":true},{"text":" is non-commercial, exactly as lesson two warned. If the output is going anywhere near a paying client, this is not the model. "},{"text":"/learn/ai-images","href":"https://addaly.com/learn/ai-images"},{"text":" covers the craft side."}]},{"t":"p","runs":[{"text":"Chat models run through llama.cpp, not through ","b":true},{"text":"transformers","b":true,"code":true},{"text":".","b":true},{"text":" The GGUF format packs quantised weights into a single file, and the Hub is full of them from community quantisers. "},{"text":"Q4_K_M","code":true},{"text":" is the usual balance of size against damage. Ollama and LM Studio both pull GGUF files straight from the Hub, and LM Studio has a graphical interface on Windows, macOS and Linux, so this path needs no Python at all. Rough expectations: with 8 GB of RAM a 3B model at 4-bit reads at a comfortable speed and a 7B is slow but usable; with 16 GB, 7B to 8B is comfortable. Apple silicon does unusually well because the CPU and GPU share memory. "},{"text":"/learn/running-models-yourself","href":"https://addaly.com/learn/running-models-yourself"},{"text":" goes into this properly."}]},{"t":"p","runs":[{"text":"A phone does not run these.","b":true},{"text":" What a phone runs is a browser pointed at a Space, or very small models compiled for the browser through transformers.js. That is not a limitation to work around; it is the reason lessons four and five exist."}]},{"t":"h2","id":"do-this-now","runs":[{"text":"Do this now"}],"role":"try"},{"t":"p","runs":[{"text":"Run "},{"text":"hf cache scan","code":true},{"text":" on whatever machine you have been experimenting on. Most people find several gigabytes they did not know about, and at least one model downloaded twice."}]}],"check":{"question":[{"text":"A laptop is out of disk. The user opens the Hugging Face cache, finds a "},{"text":"snapshots","code":true},{"text":" folder full of files like "},{"text":"model.safetensors","code":true},{"text":", deletes several gigabytes of them, and the free space barely changes. Why?"}],"options":[{"text":[{"text":"The deleted files went to the operating system’s trash and still occupy the disk until it is emptied."}],"correct":false,"tempting":[{"text":"Reaches for a general desktop behaviour instead of the cache’s own layout."}]},{"text":[{"text":"The library re-downloaded the models automatically as soon as it noticed they were missing."}],"correct":false,"tempting":[{"text":"Assumes an active repair process rather than a passive cache that only fetches when asked."}]},{"text":[{"text":"The files under "},{"text":"snapshots","code":true},{"text":" are symbolic links to blobs stored elsewhere in the cache, so removing them deletes the names and not the bytes."}],"correct":true,"tempting":null},{"text":[{"text":"Those were quantised copies that share weights with the full-precision originals, so the real space is held by the originals."}],"correct":false,"tempting":[{"text":"Invents weight sharing between separate model files instead of the deduplication the cache actually does at the blob level."}]}],"order":[3,2,1,0],"why":null},"prev":{"slug":"loads-but-outputs-rubbish","title":"It loaded, it ran, the output is nonsense","minutes":9},"next":{"slug":"dtype-and-the-memory-bill","title":"The memory arithmetic you can do in your head","minutes":9}}}