The card is where a dataset stops being a file

What a dataset card has to answer

A model card answers “what will this do”. A dataset card answers something harder: “where did this come from, and what am I allowed and likely to get wrong with it”. Four questions carry most of the weight.

Where did the rows come from? Scraped, licensed, generated by a model, written by annotators, donated by users. This single sentence determines legality, bias and whether a benchmark is contaminated. A card that will not say is telling you something.

What is the licence, and does it match the source? A dataset scraped from a site with restrictive terms does not become permissive because its uploader ticked Apache 2.0. The rights a dataset can grant are bounded by the rights its compiler held. Where a card’s licence and its described origin disagree, the disagreement is the finding.

Who or what labelled it? Crowd workers paid per item, domain experts, automatic rules, or another model. Each produces a different error profile. Labels from a model carry that model’s biases into anything you train, and if the labelling model is the one you later benchmark against, your evaluation is circular.

What is in it that you would not expect? Duplicates, a dominant language, a date range, a skewed source mix, offensive content. The best cards state this plainly; most do not state it at all.

The structured part

The front matter is YAML, and it is what the Hub indexes:

---
license: cc-by-4.0
language:
  - hi
  - en
task_categories:
  - text-classification
size_categories:
  - 10K<n<100K
annotations_creators:
  - expert-generated
source_datasets:
  - original
configs:
  - config_name: default
    data_files:
      - split: train
        path: data/train-*.parquet
---

configs is the functional part: it tells load_dataset which files belong to which split, which is why a well-configured repository loads with no arguments.

Writing the prose half

A usable card, in order:

1. One sentence saying what a row is. “Each row is a customer support message in Hindi or English with an intent label from a fixed set of nine.”
2. Collection. When, from where, by what method, over what period.
3. Labelling. Who, with what instructions, and (the number almost nobody publishes) the inter-annotator agreement. If two labellers agreed on 78% of items, no model trained on it should be expected to exceed that by much, and knowing 78% before you start saves you weeks of chasing a ceiling.
4. Splits, with counts and how they were made. Say if the split is random, and say if it is by time or by document, because that choice decides whether your evaluation is honest.
5. Known problems. Duplicates left in, a class that is mostly one source, text that is machine-translated, a date range that stops.
6. Personal data, and what was done about it.
7. What it should not be used for. The most useful section and the rarest.

The contamination note

If your dataset contains text that appears in common web crawls, and almost anything published before 2023 does, then models trained on the web may have seen it. Say so. A benchmark that has leaked into pretraining corpora still has uses, and reporting a number on it as though it were held out is misleading. This is unresolved across the field, not a solved problem with a checkbox, and the honest position is disclosure rather than a claim of cleanliness.

The legal shape, stated honestly

Whether training a model on copyrighted material is permitted differs by country and is actively contested. Several jurisdictions have text-and-data-mining exceptions with conditions attached; others do not; litigation is ongoing in more than one. Nothing here is legal advice and nobody writing a dataset card can resolve it.

What you can do is make the question answerable by somebody qualified: record the source of every row, record the terms under which you obtained it, keep a source column, and keep the date. A dataset whose provenance is documented can be assessed later under whatever the law turns out to be. A dataset whose provenance was never recorded cannot be, and that is the failure that closes doors.

Do this now

Open the card of a dataset you have used and try to answer the four questions at the top of this lesson. If you cannot answer two of them, you do not know what you trained on, and that is a finding worth writing down before the next run.