Hugging Face as Infrastructure

Hugging Face is not a website with download buttons. It is a git host with large-file storage, an API, and a set of conventions. Once you see it that way, most of it becomes predictable.

A repo is a git repo

Every model is a git repository with git-lfs for the big files. It has branches, tags, and commit hashes. main moves. People push new quantizations, fix a broken chat template, or silently change a config.

So pin the revision:

pip install -U "huggingface_hub[cli]"
hf auth login

hf download Qwen/Qwen3-8B \
  --revision 9a3f21c \
  --include "*.safetensors" "config.json" "tokenizer*" \
  --local-dir ./models/qwen3-8b

The same in code: from_pretrained("Qwen/Qwen3-8B", revision="9a3f21c"). A pinned revision is the difference between a build you can rerun in six months and a build you cannot.

Three environment variables worth knowing, especially on a shared or small machine:

export HF_HOME=/data/hf                  # cache somewhere with room
export HF_HUB_ENABLE_HF_TRANSFER=1       # faster parallel downloads
export HF_ENDPOINT=https://hf-mirror.com # if the main host is slow or blocked where you are

And clean up, because the cache silently grows to hundreds of gigabytes:

hf cache scan

The model card is a statement of intent

The README has YAML front matter (license, base_model, language, pipeline_tag, tags) and then prose. Read for six things: context length, the chat template, intended and out-of-scope use, what languages are claimed, what evaluations were run, and stated limitations.

A card that does not state a context length or show a chat template tells you something about how carefully the release was made. That is useful information even before you load anything.

The one hard rule about file formats

.safetensors is a JSON header describing tensor shapes and offsets, followed by raw bytes. There is no code in it. Loading it cannot execute anything. It also memory-maps, so it loads faster.

.bin, .pt, .pth, .ckpt are usually Python pickle. Unpickling runs code by design: that is what the format does. A crafted file opens a shell on the machine that loads it. This is not theoretical; malicious models have been found on public hubs. Hugging Face scans uploads, which is a filter, not a proof.

.gguf is llama.cpp’s single-file format, carrying weights plus metadata plus the tokenizer. Data, not code. Parsers have had memory-safety bugs, so it is not magic, but it is in a different class from pickle.

In practice: if a repo has safetensors, use them. If it only has .bin, either find a converted mirror, convert it yourself inside a container with no network and no credentials, or skip the model. And keep PyTorch’s safe default:

import torch
sd = torch.load("pytorch_model.bin", weights_only=True)  # refuses arbitrary code; the default since torch 2.6

If loading fails with weights_only=True, that is a signal, not an inconvenience to switch off.

Sizes, so you can plan a download

Before you start a 140GB transfer on a metered connection, do the arithmetic. Roughly two bytes per parameter at bf16:

Fetch only what you need:

hf download bartowski/Qwen2.5-7B-Instruct-GGUF \
  --include "*Q4_K_M.gguf" --local-dir ./gguf

The rest of the hub

Datasets are the same git plus parquet arrangement. Spaces are hosted demos, which are useful for trying a model for thirty seconds before committing to a download. Inference providers route API calls to third parties serving those same open weights, which is how you test six models without downloading any of them, the subject of the last lesson.