{"course":{"slug":"running-models-yourself","title":"Running Models Yourself","url":"https://addaly.com/learn/running-models-yourself","level":"intermediate","track":"deep"},"lesson":{"slug":"downloading-models-properly","title":"Downloading models without wasting a day","url":"https://addaly.com/learn/running-models-yourself/downloading-models-properly","position":31,"of":82,"minutes":8,"answer":"Download one quant with an include filter, pin the revision when the result matters.","keyIdea":"Download one quant with an include filter, pin the revision when the result matters, and check the two config files before committing to gigabytes: model repositories are mutable and most of them contain several copies of the same model."},"body":{"format":"markdown","text":"## Fetching only what you need\n\nA model repository holds several complete copies of the model: the original weights, sometimes a duplicate in an older format, sometimes several quants. Cloning it naively downloads all of them. The tooling lets you be specific:\n\n```bash\npip install -U \"huggingface_hub[cli]\" hf_transfer\nexport HF_HUB_ENABLE_HF_TRANSFER=1\n\n# just one quant from a GGUF repository\nhf download bartowski/Qwen3-8B-GGUF \\\n  --include \"*Q4_K_M*\" --local-dir ~/models/qwen3-8b\n\n# a full model, excluding a duplicate format\nhf download Qwen/Qwen3-8B --exclude \"*.pth\" \"*.msgpack\" \\\n  --local-dir ~/models/Qwen3-8B\n```\n\n`hf_transfer` is a Rust downloader that parallelises across connections. On a fast link it is several times quicker; on a slow one it makes no difference and can be less resilient, so turn it off if you are seeing failures.\n\nDownloads resume. If a 40 GB fetch dies at 80%, run the same command again rather than starting over.\n\n## Where files actually land\n\nBy default everything goes into a shared cache, typically `~/.cache/huggingface/hub`, with content-addressed blobs and symlinks pointing at them. That is deliberate: two models sharing a tokeniser store it once, and re-downloading a revision you already have is free.\n\nIt also surprises people. `--local-dir` puts real files where you asked and, in recent versions, keeps them independent of the cache. Set `HF_HOME` to move the whole cache to a larger disk:\n\n```bash\nexport HF_HOME=/mnt/big/hf\n```\n\nAnd clean up deliberately, because this directory grows without bound:\n\n```bash\nhf cache scan\nhf cache delete\n```\n\nOllama keeps its own store, usually `~/.ollama/models`, in a layered format. `ollama list` and `ollama rm` manage it, and `OLLAMA_MODELS` relocates it.\n\n## Pin the revision\n\nModel repositories are mutable. Weights get re-uploaded after a bug fix, templates get corrected, and occasionally a model is replaced with a different one under the same name. If a result matters, pin it:\n\n```bash\nhf download Qwen/Qwen3-8B --revision <commit-sha> --local-dir ./pinned\n```\n\nThis is the difference between a reproducible setup and one that changes under you. It costs nothing at download time and saves the argument about whether the model changed or your prompt did.\n\n## Verify what arrived\n\nEvery file on the hub carries a hash, and the download tools check it. If you fetched by other means (a mirror, a colleague's drive, a torrent) verify before running:\n\n```bash\nsha256sum model.gguf\n```\n\nA truncated GGUF often loads and then produces nonsense partway through, which is a confusing failure. Comparing one hash removes the possibility.\n\n## On a slow or metered connection\n\nThis is most of the world and the documentation rarely acknowledges it.\n\n- **Check the file size before starting.** The repository lists it. A 4.9 GB file on a 5 Mbit connection is two and a half hours.\n- **Download the two small config files first** and do the memory arithmetic. Two kilobytes to avoid five gigabytes of mistake.\n- **Prefer one quant.** People download three tiers to compare and then never compare them.\n- **Use a mirror if the main host is slow or blocked from where you are.** Regional mirrors exist and are configured with an environment variable, `HF_ENDPOINT`. Verify the hash afterwards.\n- **Move models between machines on a USB drive.** A GGUF is a single self-contained file; copying it is legitimate and it works. Nothing needs to be re-downloaded.\n\n## When the connection is the problem\n\nTwo environment variables solve most download misery and almost nobody knows about them.\n\n```bash\npip install hf_transfer\nexport HF_HUB_ENABLE_HF_TRANSFER=1   # multi-threaded, saturates a fast link\nexport HF_ENDPOINT=https://<mirror>  # route via a mirror where the hub is slow\n```\n\nThe first replaces the default single-stream download with a parallel one. On a connection above roughly 200 Mbit/s the difference is several times faster, because the bottleneck was never the link. The trade is that its progress reporting is cruder and a failure is less informative, so turn it off when you are debugging a download rather than doing one.\n\nThe second matters in regions where the hub is slow or intermittently unreachable. Community mirrors exist and serve the same files; verify the checksums afterwards exactly as you would otherwise, because a mirror is a third party you have chosen to trust.\n\nBoth the hub client and `curl -C -` resume from where they stopped, so a 40 GB pull interrupted at 30 GB continues rather than restarts. Interrupt it deliberately if you need the bandwidth; nothing is lost.\n\n## Disk planning\n\nA modest working set fills a disk faster than expected: an 8B at Q4 is 4.9 GB, a 14B is 8.5, a 32B is 19, a 70B is 42. Three models and two quants each is well over 100 GB.\n\nKeep them on a drive with room to spare, know where the cache is, and scan it every few months. The alternative is discovering the problem when a system update fails for want of space, which is how most people learn where the cache lives."},"check":{"question":"A team's evaluation results shift noticeably a month after they were recorded, with no change to their code, prompts or engine version. What is the most likely cause, and what would have prevented it?","options":[{"text":"The cache was garbage-collected and the model silently re-downloaded at a lower precision; pinning the quant filename would have prevented it.","correct":false,"tempting":"Invents a behaviour caches do not have: they never substitute a different file for the one requested."},{"text":"The model repository was updated in place and their unpinned download fetched a newer revision; pinning a commit SHA would have prevented it.","correct":true},{"text":"The tokeniser was shared with another model in the content-addressed cache and was overwritten by the other model's version.","correct":false,"tempting":"Misunderstands content addressing: identical content shares a blob, and differing content gets a different blob, so overwriting cannot occur."},{"text":"Model files degrade on disk over time and the checksums no longer match; regular verification would have caught it.","correct":false,"tempting":"Reaches for bit rot to explain a coherent behavioural change, which corruption does not produce."}]},"prev":{"slug":"installing-the-right-build","title":"Installing the right build for your hardware","url":"https://addaly.com/learn/running-models-yourself/installing-the-right-build"},"next":{"slug":"chat-templates","title":"Chat templates: the most common silent failure","url":"https://addaly.com/learn/running-models-yourself/chat-templates"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Downloading models without wasting a day', in Running Models Yourself. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/running-models-yourself/downloading-models-properly (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Downloading models without wasting a day","courseTitle":"Running Models Yourself","module":{"index":4,"slug":"getting-it-running","title":"Getting it running"},"answer":null,"keyIdea":"Download one quant with an include filter, pin the revision when the result matters, and check the two config files before committing to gigabytes: model repositories are mutable and most of them contain several copies of the same model.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"fetching-only-what-you-need","runs":[{"text":"Fetching only what you need"}]},{"t":"p","runs":[{"text":"A model repository holds several complete copies of the model: the original weights, sometimes a duplicate in an older format, sometimes several quants. Cloning it naively downloads all of them. The tooling lets you be specific:"}]},{"t":"code","lang":"bash","label":"Bash","text":"pip install -U \"huggingface_hub[cli]\" hf_transfer\nexport HF_HUB_ENABLE_HF_TRANSFER=1\n\n# just one quant from a GGUF repository\nhf download bartowski/Qwen3-8B-GGUF \\\n  --include \"*Q4_K_M*\" --local-dir ~/models/qwen3-8b\n\n# a full model, excluding a duplicate format\nhf download Qwen/Qwen3-8B --exclude \"*.pth\" \"*.msgpack\" \\\n  --local-dir ~/models/Qwen3-8B"},{"t":"p","runs":[{"text":"hf_transfer","code":true},{"text":" is a Rust downloader that parallelises across connections. On a fast link it is several times quicker; on a slow one it makes no difference and can be less resilient, so turn it off if you are seeing failures."}]},{"t":"p","runs":[{"text":"Downloads resume. If a 40 GB fetch dies at 80%, run the same command again rather than starting over."}]},{"t":"h2","id":"where-files-actually-land","runs":[{"text":"Where files actually land"}]},{"t":"p","runs":[{"text":"By default everything goes into a shared cache, typically "},{"text":"~/.cache/huggingface/hub","code":true},{"text":", with content-addressed blobs and symlinks pointing at them. That is deliberate: two models sharing a tokeniser store it once, and re-downloading a revision you already have is free."}]},{"t":"p","runs":[{"text":"It also surprises people. "},{"text":"--local-dir","code":true},{"text":" puts real files where you asked and, in recent versions, keeps them independent of the cache. Set "},{"text":"HF_HOME","code":true},{"text":" to move the whole cache to a larger disk:"}]},{"t":"code","lang":"bash","label":"Bash","text":"export HF_HOME=/mnt/big/hf"},{"t":"p","runs":[{"text":"And clean up deliberately, because this directory grows without bound:"}]},{"t":"code","lang":"bash","label":"Bash","text":"hf cache scan\nhf cache delete"},{"t":"p","runs":[{"text":"Ollama keeps its own store, usually "},{"text":"~/.ollama/models","code":true},{"text":", in a layered format. "},{"text":"ollama list","code":true},{"text":" and "},{"text":"ollama rm","code":true},{"text":" manage it, and "},{"text":"OLLAMA_MODELS","code":true},{"text":" relocates it."}]},{"t":"h2","id":"pin-the-revision","runs":[{"text":"Pin the revision"}]},{"t":"p","runs":[{"text":"Model repositories are mutable. Weights get re-uploaded after a bug fix, templates get corrected, and occasionally a model is replaced with a different one under the same name. If a result matters, pin it:"}]},{"t":"code","lang":"bash","label":"Bash","text":"hf download Qwen/Qwen3-8B --revision <commit-sha> --local-dir ./pinned"},{"t":"p","runs":[{"text":"This is the difference between a reproducible setup and one that changes under you. It costs nothing at download time and saves the argument about whether the model changed or your prompt did."}]},{"t":"h2","id":"verify-what-arrived","runs":[{"text":"Verify what arrived"}]},{"t":"p","runs":[{"text":"Every file on the hub carries a hash, and the download tools check it. If you fetched by other means (a mirror, a colleague’s drive, a torrent) verify before running:"}]},{"t":"code","lang":"bash","label":"Bash","text":"sha256sum model.gguf"},{"t":"p","runs":[{"text":"A truncated GGUF often loads and then produces nonsense partway through, which is a confusing failure. Comparing one hash removes the possibility."}]},{"t":"h2","id":"on-a-slow-or-metered-connection","runs":[{"text":"On a slow or metered connection"}]},{"t":"p","runs":[{"text":"This is most of the world and the documentation rarely acknowledges it."}]},{"t":"ul","items":[[{"text":"Check the file size before starting.","b":true},{"text":" The repository lists it. A 4.9 GB file on a 5 Mbit connection is two and a half hours."}],[{"text":"Download the two small config files first","b":true},{"text":" and do the memory arithmetic. Two kilobytes to avoid five gigabytes of mistake."}],[{"text":"Prefer one quant.","b":true},{"text":" People download three tiers to compare and then never compare them."}],[{"text":"Use a mirror if the main host is slow or blocked from where you are.","b":true},{"text":" Regional mirrors exist and are configured with an environment variable, "},{"text":"HF_ENDPOINT","code":true},{"text":". Verify the hash afterwards."}],[{"text":"Move models between machines on a USB drive.","b":true},{"text":" A GGUF is a single self-contained file; copying it is legitimate and it works. Nothing needs to be re-downloaded."}]]},{"t":"h2","id":"when-the-connection-is-the-problem","runs":[{"text":"When the connection is the problem"}]},{"t":"p","runs":[{"text":"Two environment variables solve most download misery and almost nobody knows about them."}]},{"t":"code","lang":"bash","label":"Bash","text":"pip install hf_transfer\nexport HF_HUB_ENABLE_HF_TRANSFER=1   # multi-threaded, saturates a fast link\nexport HF_ENDPOINT=https://<mirror>  # route via a mirror where the hub is slow"},{"t":"p","runs":[{"text":"The first replaces the default single-stream download with a parallel one. On a connection above roughly 200 Mbit/s the difference is several times faster, because the bottleneck was never the link. The trade is that its progress reporting is cruder and a failure is less informative, so turn it off when you are debugging a download rather than doing one."}]},{"t":"p","runs":[{"text":"The second matters in regions where the hub is slow or intermittently unreachable. Community mirrors exist and serve the same files; verify the checksums afterwards exactly as you would otherwise, because a mirror is a third party you have chosen to trust."}]},{"t":"p","runs":[{"text":"Both the hub client and "},{"text":"curl -C -","code":true},{"text":" resume from where they stopped, so a 40 GB pull interrupted at 30 GB continues rather than restarts. Interrupt it deliberately if you need the bandwidth; nothing is lost."}]},{"t":"h2","id":"disk-planning","runs":[{"text":"Disk planning"}]},{"t":"p","runs":[{"text":"A modest working set fills a disk faster than expected: an 8B at Q4 is 4.9 GB, a 14B is 8.5, a 32B is 19, a 70B is 42. Three models and two quants each is well over 100 GB."}]},{"t":"p","runs":[{"text":"Keep them on a drive with room to spare, know where the cache is, and scan it every few months. The alternative is discovering the problem when a system update fails for want of space, which is how most people learn where the cache lives."}]}],"check":{"question":[{"text":"A team’s evaluation results shift noticeably a month after they were recorded, with no change to their code, prompts or engine version. What is the most likely cause, and what would have prevented it?"}],"options":[{"text":[{"text":"The cache was garbage-collected and the model silently re-downloaded at a lower precision; pinning the quant filename would have prevented it."}],"correct":false,"tempting":[{"text":"Invents a behaviour caches do not have: they never substitute a different file for the one requested."}]},{"text":[{"text":"The model repository was updated in place and their unpinned download fetched a newer revision; pinning a commit SHA would have prevented it."}],"correct":true,"tempting":null},{"text":[{"text":"The tokeniser was shared with another model in the content-addressed cache and was overwritten by the other model’s version."}],"correct":false,"tempting":[{"text":"Misunderstands content addressing: identical content shares a blob, and differing content gets a different blob, so overwriting cannot occur."}]},{"text":[{"text":"Model files degrade on disk over time and the checksums no longer match; regular verification would have caught it."}],"correct":false,"tempting":[{"text":"Reaches for bit rot to explain a coherent behavioural change, which corruption does not produce."}]}],"order":[2,3,1,0],"why":null},"prev":{"slug":"installing-the-right-build","title":"Installing the right build for your hardware","minutes":9},"next":{"slug":"chat-templates","title":"Chat templates: the most common silent failure","minutes":10}}}