{"course":{"slug":"hugging-face","title":"Hugging Face, End to End","url":"https://addaly.com/learn/hugging-face","level":"intermediate","track":"create"},"lesson":{"slug":"the-hub-api","title":"Ask the Hub questions instead of browsing it","url":"https://addaly.com/learn/hugging-face/the-hub-api","position":8,"of":75,"minutes":9,"answer":"huggingface_hub turns the Hub into a queryable service, and the single highest-value call is model_info(..., files_metadata=True).","keyIdea":"`huggingface_hub` turns the Hub into a queryable service, and the single highest-value call is `model_info(..., files_metadata=True)`, which tells you exactly what a repository will cost in disk and what formats it duplicates before you spend a byte."},"body":{"format":"markdown","text":"## The website is one client of an API\n\nEverything the Hub shows you is available as JSON, and the `huggingface_hub` package wraps it. This matters for three jobs the browser is bad at: answering questions about many repositories at once, checking things before you download them, and automating the boring parts of publishing.\n\n```bash\npip install -U huggingface_hub\n```\n\n## Listing models with the filters you would have clicked\n\n```python\nfrom huggingface_hub import HfApi\n\napi = HfApi()\n\nmodels = api.list_models(\n    task=\"automatic-speech-recognition\",\n    library=\"transformers\",\n    language=\"hi\",\n    sort=\"downloads\",\n    direction=-1,\n    limit=20,\n)\n\nfor m in models:\n    print(f\"{m.downloads:>9,}  {m.id}\")\n```\n\nThis is the URL from the search lesson, in code, and it returns objects you can filter further in Python, which is where it stops being a browser replacement and starts being useful. \"Speech models for Hindi, updated in the last year, under Apache 2.0\" is three lines of list comprehension and is not a query the sidebar can express.\n\n## Checking size before you download\n\nThe most valuable call is the one that costs nothing:\n\n```python\ninfo = api.model_info(\"openai/whisper-large-v3\", files_metadata=True)\n\ntotal = sum(f.size or 0 for f in info.siblings)\nprint(f\"{len(info.siblings)} files, {total / 1e9:.1f} GB\")\nfor f in sorted(info.siblings, key=lambda s: -(s.size or 0))[:5]:\n    print(f\"  {(f.size or 0)/1e9:6.2f} GB  {f.rfilename}\")\n```\n\nRun that before any `snapshot_download` and you will stop being surprised by disk usage. It also shows the duplicate-format problem from the last lesson immediately: you will see the `.bin` and the `.safetensors` sitting next to each other, each several gigabytes.\n\n`model_info` also carries `info.tags`, `info.cardData` (the parsed YAML front matter of the card, including licence and `base_model`), `info.gated`, and `info.lastModified`. Everything the card page shows in a badge is in there as a field.\n\n## Downloading precisely\n\n```python\nfrom huggingface_hub import hf_hub_download, snapshot_download\n\n# one file\ncfg = hf_hub_download(\"openai/whisper-large-v3\", \"config.json\")\n\n# a subset, pinned, skipping duplicate formats\npath = snapshot_download(\n    \"openai/whisper-large-v3\",\n    revision=\"06f233fe06e710322aca913c1bc4249a0d71fce1\",\n    allow_patterns=[\"*.safetensors\", \"*.json\", \"*.txt\"],\n    ignore_patterns=[\"*.bin\", \"onnx/*\"],\n)\n```\n\n`allow_patterns` and `ignore_patterns` are the difference between a 6 GB download and a 12 GB one on repositories that ship several formats. Set `HF_HUB_ENABLE_HF_TRANSFER=1` in the environment after `pip install hf_transfer` if your connection is fast enough to be bottlenecked by Python rather than the network; on a slow link it makes no difference.\n\n## Writing: repositories, files, and the commit you can see\n\n```python\napi.create_repo(\"your-name/your-model\", repo_type=\"model\", private=True)\n\napi.upload_file(\n    path_or_fileobj=\"README.md\",\n    path_in_repo=\"README.md\",\n    repo_id=\"your-name/your-model\",\n    commit_message=\"Add model card with licence and limitations\",\n)\n\napi.upload_folder(\n    folder_path=\"./out\",\n    repo_id=\"your-name/your-model\",\n    ignore_patterns=[\"*.ckpt\", \"runs/*\", \"*.log\"],\n)\n```\n\nEvery write is a commit with a message, visible in History. Use the message: six months later it is the only record of why a checkpoint changed.\n\n## The command line does the same things\n\n```bash\nhf auth login\nhf download openai/whisper-small --include \"*.safetensors\" \"*.json\"\nhf upload your-name/your-model ./out --commit-message \"first checkpoint\"\nhf repo create your-name/your-dataset --repo-type dataset\nhf cache scan\n```\n\nThe binary was called `huggingface-cli` until 2025 and both names still appear in tutorials. `hf cache scan` is the one to remember; it prints what your cache is holding and is the quickest way to find the 40 GB you cannot account for.\n\n## What the API will not give you\n\n- **Rate limits are real.** Anonymous calls are limited more tightly than authenticated ones, and a loop over ten thousand `model_info` calls will start getting HTTP 429. Authenticate, batch, and cache results to disk.\n- **`downloads` is a thirty-day count**, not a total, and the next lesson explains what it counts.\n- **The API cannot tell you whether a model is good.** Every field is either a counter or something the uploader typed. There is no quality signal in the JSON, and treating `downloads` as one is the mistake this whole module exists to prevent.\n\n## Do this now\n\nWrite ten lines that list the twenty most-downloaded models for a task you care about, and print each one's licence from `cardData`. You will find that several of the top results have no licence field at all, which, as the card lesson argued, is not the same as permissive."},"check":{"question":"Why does `model_info(repo, files_metadata=True)` frequently report a total far larger than what you need to download?","options":[{"text":"It reports the size of the full commit history, including superseded checkpoint versions still held in large-file storage.","correct":false,"tempting":"Confuses repository storage with the current revision's file listing, which is what siblings describes."},{"text":"It counts every file at that revision, including duplicate weight formats you would never fetch.","correct":true},{"text":"Sizes are returned before large-file deduplication, so the true transfer is always substantially smaller than reported.","correct":false,"tempting":"Deduplication affects upload and storage, not the bytes a fresh client must receive."},{"text":"The call sums sizes across every branch and open pull request in the repository.","correct":false,"tempting":"Assumes a repository-wide aggregate when the call is scoped to one revision."}]},"prev":{"slug":"four-shelves","title":"Four shelves, and picking the wrong one costs a weekend","url":"https://addaly.com/learn/hugging-face/four-shelves"},"next":{"slug":"counters-and-trending","title":"What a download count is counting","url":"https://addaly.com/learn/hugging-face/counters-and-trending"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Ask the Hub questions instead of browsing it', in Hugging Face, End to End. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/hugging-face/the-hub-api (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Ask the Hub questions instead of browsing it","courseTitle":"Hugging Face, End to End","module":{"index":1,"slug":"the-hub-as-a-repository-host","title":"The Hub, read properly"},"answer":null,"keyIdea":"`huggingface_hub` turns the Hub into a queryable service, and the single highest-value call is `model_info(..., files_metadata=True)`, which tells you exactly what a repository will cost in disk and what formats it duplicates before you spend a byte.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-website-is-one-client-of-an-api","runs":[{"text":"The website is one client of an API"}]},{"t":"p","runs":[{"text":"Everything the Hub shows you is available as JSON, and the "},{"text":"huggingface_hub","code":true},{"text":" package wraps it. This matters for three jobs the browser is bad at: answering questions about many repositories at once, checking things before you download them, and automating the boring parts of publishing."}]},{"t":"code","lang":"bash","label":"Bash","text":"pip install -U huggingface_hub"},{"t":"h2","id":"listing-models-with-the-filters-you-would-have-clicked","runs":[{"text":"Listing models with the filters you would have clicked"}]},{"t":"code","lang":"python","label":"Python","text":"from huggingface_hub import HfApi\n\napi = HfApi()\n\nmodels = api.list_models(\n    task=\"automatic-speech-recognition\",\n    library=\"transformers\",\n    language=\"hi\",\n    sort=\"downloads\",\n    direction=-1,\n    limit=20,\n)\n\nfor m in models:\n    print(f\"{m.downloads:>9,}  {m.id}\")"},{"t":"p","runs":[{"text":"This is the URL from the search lesson, in code, and it returns objects you can filter further in Python, which is where it stops being a browser replacement and starts being useful. “Speech models for Hindi, updated in the last year, under Apache 2.0” is three lines of list comprehension and is not a query the sidebar can express."}]},{"t":"h2","id":"checking-size-before-you-download","runs":[{"text":"Checking size before you download"}]},{"t":"p","runs":[{"text":"The most valuable call is the one that costs nothing:"}]},{"t":"code","lang":"python","label":"Python","text":"info = api.model_info(\"openai/whisper-large-v3\", files_metadata=True)\n\ntotal = sum(f.size or 0 for f in info.siblings)\nprint(f\"{len(info.siblings)} files, {total / 1e9:.1f} GB\")\nfor f in sorted(info.siblings, key=lambda s: -(s.size or 0))[:5]:\n    print(f\"  {(f.size or 0)/1e9:6.2f} GB  {f.rfilename}\")"},{"t":"p","runs":[{"text":"Run that before any "},{"text":"snapshot_download","code":true},{"text":" and you will stop being surprised by disk usage. It also shows the duplicate-format problem from the last lesson immediately: you will see the "},{"text":".bin","code":true},{"text":" and the "},{"text":".safetensors","code":true},{"text":" sitting next to each other, each several gigabytes."}]},{"t":"p","runs":[{"text":"model_info","code":true},{"text":" also carries "},{"text":"info.tags","code":true},{"text":", "},{"text":"info.cardData","code":true},{"text":" (the parsed YAML front matter of the card, including licence and "},{"text":"base_model","code":true},{"text":"), "},{"text":"info.gated","code":true},{"text":", and "},{"text":"info.lastModified","code":true},{"text":". Everything the card page shows in a badge is in there as a field."}]},{"t":"h2","id":"downloading-precisely","runs":[{"text":"Downloading precisely"}]},{"t":"code","lang":"python","label":"Python","text":"from huggingface_hub import hf_hub_download, snapshot_download\n\n# one file\ncfg = hf_hub_download(\"openai/whisper-large-v3\", \"config.json\")\n\n# a subset, pinned, skipping duplicate formats\npath = snapshot_download(\n    \"openai/whisper-large-v3\",\n    revision=\"06f233fe06e710322aca913c1bc4249a0d71fce1\",\n    allow_patterns=[\"*.safetensors\", \"*.json\", \"*.txt\"],\n    ignore_patterns=[\"*.bin\", \"onnx/*\"],\n)"},{"t":"p","runs":[{"text":"allow_patterns","code":true},{"text":" and "},{"text":"ignore_patterns","code":true},{"text":" are the difference between a 6 GB download and a 12 GB one on repositories that ship several formats. Set "},{"text":"HF_HUB_ENABLE_HF_TRANSFER=1","code":true},{"text":" in the environment after "},{"text":"pip install hf_transfer","code":true},{"text":" if your connection is fast enough to be bottlenecked by Python rather than the network; on a slow link it makes no difference."}]},{"t":"h2","id":"writing-repositories-files-and-the-commit-you-can-see","runs":[{"text":"Writing: repositories, files, and the commit you can see"}]},{"t":"code","lang":"python","label":"Python","text":"api.create_repo(\"your-name/your-model\", repo_type=\"model\", private=True)\n\napi.upload_file(\n    path_or_fileobj=\"README.md\",\n    path_in_repo=\"README.md\",\n    repo_id=\"your-name/your-model\",\n    commit_message=\"Add model card with licence and limitations\",\n)\n\napi.upload_folder(\n    folder_path=\"./out\",\n    repo_id=\"your-name/your-model\",\n    ignore_patterns=[\"*.ckpt\", \"runs/*\", \"*.log\"],\n)"},{"t":"p","runs":[{"text":"Every write is a commit with a message, visible in History. Use the message: six months later it is the only record of why a checkpoint changed."}]},{"t":"h2","id":"the-command-line-does-the-same-things","runs":[{"text":"The command line does the same things"}]},{"t":"code","lang":"bash","label":"Bash","text":"hf auth login\nhf download openai/whisper-small --include \"*.safetensors\" \"*.json\"\nhf upload your-name/your-model ./out --commit-message \"first checkpoint\"\nhf repo create your-name/your-dataset --repo-type dataset\nhf cache scan"},{"t":"p","runs":[{"text":"The binary was called "},{"text":"huggingface-cli","code":true},{"text":" until 2025 and both names still appear in tutorials. "},{"text":"hf cache scan","code":true},{"text":" is the one to remember; it prints what your cache is holding and is the quickest way to find the 40 GB you cannot account for."}]},{"t":"h2","id":"what-the-api-will-not-give-you","runs":[{"text":"What the API will not give you"}]},{"t":"ul","items":[[{"text":"Rate limits are real.","b":true},{"text":" Anonymous calls are limited more tightly than authenticated ones, and a loop over ten thousand "},{"text":"model_info","code":true},{"text":" calls will start getting HTTP 429. Authenticate, batch, and cache results to disk."}],[{"text":"downloads","b":true,"code":true},{"text":" is a thirty-day count","b":true},{"text":", not a total, and the next lesson explains what it counts."}],[{"text":"The API cannot tell you whether a model is good.","b":true},{"text":" Every field is either a counter or something the uploader typed. There is no quality signal in the JSON, and treating "},{"text":"downloads","code":true},{"text":" as one is the mistake this whole module exists to prevent."}]]},{"t":"h2","id":"do-this-now","runs":[{"text":"Do this now"}],"role":"try"},{"t":"p","runs":[{"text":"Write ten lines that list the twenty most-downloaded models for a task you care about, and print each one’s licence from "},{"text":"cardData","code":true},{"text":". You will find that several of the top results have no licence field at all, which, as the card lesson argued, is not the same as permissive."}]}],"check":{"question":[{"text":"Why does "},{"text":"model_info(repo, files_metadata=True)","code":true},{"text":" frequently report a total far larger than what you need to download?"}],"options":[{"text":[{"text":"It reports the size of the full commit history, including superseded checkpoint versions still held in large-file storage."}],"correct":false,"tempting":[{"text":"Confuses repository storage with the current revision’s file listing, which is what siblings describes."}]},{"text":[{"text":"It counts every file at that revision, including duplicate weight formats you would never fetch."}],"correct":true,"tempting":null},{"text":[{"text":"Sizes are returned before large-file deduplication, so the true transfer is always substantially smaller than reported."}],"correct":false,"tempting":[{"text":"Deduplication affects upload and storage, not the bytes a fresh client must receive."}]},{"text":[{"text":"The call sums sizes across every branch and open pull request in the repository."}],"correct":false,"tempting":[{"text":"Assumes a repository-wide aggregate when the call is scoped to one revision."}]}],"order":[2,0,3,1],"why":null},"prev":{"slug":"four-shelves","title":"Four shelves, and picking the wrong one costs a weekend","minutes":8},"next":{"slug":"counters-and-trending","title":"What a download count is counting","minutes":8}}}