Hugging Face as Infrastructure

Hugging Face is not a website with download buttons. It is a git host with large-file storage, an API, and a set of conventions. Once you see it that way, most of it becomes predictable.

A repo is a git repo

Every model is a git repository with git-lfs for the big files. It has branches, tags, and commit hashes. main moves. People push new quantizations, fix a broken chat template, or silently change a config.

So pin the revision:

pip install -U "huggingface_hub[cli]"
hf auth login

hf download Qwen/Qwen3-8B \
  --revision 9a3f21c \
  --include "*.safetensors" "config.json" "tokenizer*" \
  --local-dir ./models/qwen3-8b

The same in code: from_pretrained("Qwen/Qwen3-8B", revision="9a3f21c"). A pinned revision is the difference between a build you can rerun in six months and a build you cannot.

Three environment variables worth knowing, especially on a shared or small machine:

export HF_HOME=/data/hf                  # cache somewhere with room
export HF_HUB_ENABLE_HF_TRANSFER=1       # faster parallel downloads
export HF_ENDPOINT=https://hf-mirror.com # if the main host is slow or blocked where you are

And clean up, because the cache silently grows to hundreds of gigabytes:

hf cache scan

The model card is a statement of intent

The README has YAML front matter — license, base_model, language, pipeline_tag, tags — and then prose. Read for six things: context length, the chat template, intended and out-of-scope use, what languages are claimed, what evaluations were run, and stated limitations.

A card that does not state a context length or show a chat template tells you something about how carefully the release was made. That is useful information even before you load anything.

The one hard rule about file formats

`.safetensors` is a JSON header describing tensor shapes and offsets, followed by raw bytes. There is no code in it. Loading it cannot execute anything. It also memory-maps, so it loads faster.

`.bin`, `.pt`, `.pth`, `.ckpt` are usually Python pickle. Unpickling *runs code by design* — that is what the format does. A crafted file opens a shell on the machine that loads it. This is not theoretical; malicious models have been found on public hubs. Hugging Face scans uploads, which is a filter, not a proof.

`.gguf` is llama.cpp's single-file format, carrying weights plus metadata plus the tokenizer. Data, not code. Parsers have had memory-safety bugs, so it is not magic, but it is in a different class from pickle.

In practice: if a repo has safetensors, use them. If it only has .bin, either find a converted mirror, convert it yourself inside a container with no network and no credentials, or skip the model. And keep PyTorch's safe default:

import torch
sd = torch.load("pytorch_model.bin", weights_only=True)  # refuses arbitrary code; the default since torch 2.6

If loading fails with weights_only=True, that is a signal, not an inconvenience to switch off.

Sizes, so you can plan a download

Before you start a 140GB transfer on a metered connection, do the arithmetic. Roughly two bytes per parameter at bf16:

| Model | bf16 files | Q8 GGUF | Q4_K_M GGUF | |---|---|---|---| | 4B | ~8 GB | ~4.3 GB | ~2.5 GB | | 8B | ~16 GB | ~8.5 GB | ~4.9 GB | | 27B | ~54 GB | ~29 GB | ~16 GB | | 70B | ~140 GB | ~75 GB | ~42 GB |

Fetch only what you need:

hf download bartowski/Qwen2.5-7B-Instruct-GGUF \
  --include "*Q4_K_M.gguf" --local-dir ./gguf

The rest of the hub

Datasets are the same git plus parquet arrangement. Spaces are hosted demos, which are useful for trying a model for thirty seconds before committing to a download. Inference providers route API calls to third parties serving those same open weights, which is how you test six models without downloading any of them — the subject of the last lesson.