The data card

The artefact that outlives the model

Models get replaced. Datasets get reused, extended and argued about for years. The document that makes that possible is a data card, and it takes forty minutes to write.

The practice comes from Datasheets for Datasets, which proposed that every dataset ship with a record of its motivation, composition, collection process and recommended uses: the way an electronic component ships with a datasheet. The full questionnaire is long; the version below is the part that earns its keep for an applied fine-tune.

The template

# Dataset: ticket-triage-sft, v3

## What it is
31,084 single-turn examples mapping a customer message to a triage
decision and a short reply. English only.

## Sources and counts
- support_logs   24,003  internal, ToS v4 analytics basis, 2026-01 to 2026-08
- hand_written    2,140  written by the support team against style-guide v2
- oasst1          4,941  Apache-2.0, replay mixture (16% of final)

## Filters applied, with counts
start 48,201 → exact dup -6,340 → minhash 0.85 -9,118 → short -412
→ truncated -887 → non-English -204 → repetition -156 → final 31,084

## Processing
- Presidio redaction (emails, phones, order refs, names) before storage
- 200 rows manually reviewed post-redaction; 3 leaks found and fixed
- Converted to messages format by scripts/to_messages.py @ 4f1a09

## Known gaps
- No examples of the model declining an out-of-scope request other than
  the 88 hand-written ones. Under-represented.
- Refunds are 11% of the set and 1% of traffic (deliberate, flattened).
- Nothing after 2026-08; the September pricing change is not represented.
- No non-English, though ~4% of traffic is.

## Licence and constraints
Internal use only. Contains the oasst1 subset (Apache-2.0, attribution
kept in source column). Not redistributable.

## Reproducing it
make dataset   # from raw/, deterministic given seed 0
sha256(train.jsonl) = 9f2c1e...

## Owner
r.kapoor — questions about labelling decisions go here

The four sections that do the work

Filters with counts. When somebody later asks why the model is bad at long tickets, the line truncated -887 is the first place to look. A filter list without counts is decoration.

Known gaps. The hardest section to write and the most valuable. It requires admitting what the dataset does not cover, which is exactly the information the next person needs and exactly what an enthusiastic project write-up omits. Every production surprise I have seen traced back to something that would have been in a properly written gaps section.

The hash. One line, and it is what connects a model to the data that made it. Without it, “which dataset produced this checkpoint” is answered from memory. Put the hash in the model’s own metadata too, so the link survives in both directions.

Reproducing it. If the answer is a notebook on somebody’s laptop, write that down honestly, because that is a risk the organisation should know it is carrying. If the answer is one command, you have a dataset that survives its author leaving.

Version it like code

The dataset is an input to the model, so it belongs in version control with the same discipline. Small enough files go in git directly; larger ones go in git-lfs, DVC, or a Hugging Face dataset repository, all of which are free and all of which give you a content hash.

What matters is that v3 means one specific set of bytes forever, and that a model card can name it.

Why to do this even for a project nobody else will see

Two reasons, both selfish.

The first: in four months you will be the stranger. You will not remember whether the refund flattening was deliberate, and you will spend an afternoon reconstructing it from a script.

The second: writing the gaps section changes the dataset. Nearly every time somebody sits down to list what their data does not cover, they stop halfway through and go and collect some of it. That is the document doing its job before it has been read by anybody.