{"course":{"slug":"fine-tuning","title":"Fine-Tuning, and When Not To","url":"https://addaly.com/learn/fine-tuning","level":"advanced","track":"deep"},"lesson":{"slug":"documenting-the-dataset","title":"The data card","url":"https://addaly.com/learn/fine-tuning/documenting-the-dataset","position":38,"of":84,"minutes":8,"answer":"A data card with per-filter counts, an honest known-gaps section, a content hash and a one-command rebuild is what links a model to the data that made it.","keyIdea":"A data card with per-filter counts, an honest known-gaps section, a content hash and a one-command rebuild is what links a model to the data that made it, and writing the gaps section usually changes the dataset before anybody else reads it."},"body":{"format":"markdown","text":"## The artefact that outlives the model\n\nModels get replaced. Datasets get reused, extended and argued about for years. The document that makes that possible is a data card, and it takes forty minutes to write.\n\nThe practice comes from *Datasheets for Datasets*, which proposed that every dataset ship with a record of its motivation, composition, collection process and recommended uses: the way an electronic component ships with a datasheet. The full questionnaire is long; the version below is the part that earns its keep for an applied fine-tune.\n\n## The template\n\n```markdown\n# Dataset: ticket-triage-sft, v3\n\n## What it is\n31,084 single-turn examples mapping a customer message to a triage\ndecision and a short reply. English only.\n\n## Sources and counts\n- support_logs   24,003  internal, ToS v4 analytics basis, 2026-01 to 2026-08\n- hand_written    2,140  written by the support team against style-guide v2\n- oasst1          4,941  Apache-2.0, replay mixture (16% of final)\n\n## Filters applied, with counts\nstart 48,201 → exact dup -6,340 → minhash 0.85 -9,118 → short -412\n→ truncated -887 → non-English -204 → repetition -156 → final 31,084\n\n## Processing\n- Presidio redaction (emails, phones, order refs, names) before storage\n- 200 rows manually reviewed post-redaction; 3 leaks found and fixed\n- Converted to messages format by scripts/to_messages.py @ 4f1a09\n\n## Known gaps\n- No examples of the model declining an out-of-scope request other than\n  the 88 hand-written ones. Under-represented.\n- Refunds are 11% of the set and 1% of traffic (deliberate, flattened).\n- Nothing after 2026-08; the September pricing change is not represented.\n- No non-English, though ~4% of traffic is.\n\n## Licence and constraints\nInternal use only. Contains the oasst1 subset (Apache-2.0, attribution\nkept in source column). Not redistributable.\n\n## Reproducing it\nmake dataset   # from raw/, deterministic given seed 0\nsha256(train.jsonl) = 9f2c1e...\n\n## Owner\nr.kapoor — questions about labelling decisions go here\n```\n\n## The four sections that do the work\n\n**Filters with counts.** When somebody later asks why the model is bad at long tickets, the line `truncated -887` is the first place to look. A filter list without counts is decoration.\n\n**Known gaps.** The hardest section to write and the most valuable. It requires admitting what the dataset does not cover, which is exactly the information the next person needs and exactly what an enthusiastic project write-up omits. Every production surprise I have seen traced back to something that would have been in a properly written gaps section.\n\n**The hash.** One line, and it is what connects a model to the data that made it. Without it, \"which dataset produced this checkpoint\" is answered from memory. Put the hash in the model's own metadata too, so the link survives in both directions.\n\n**Reproducing it.** If the answer is a notebook on somebody's laptop, write that down honestly, because that is a risk the organisation should know it is carrying. If the answer is one command, you have a dataset that survives its author leaving.\n\n## Version it like code\n\nThe dataset is an input to the model, so it belongs in version control with the same discipline. Small enough files go in git directly; larger ones go in git-lfs, DVC, or a Hugging Face dataset repository, all of which are free and all of which give you a content hash.\n\nWhat matters is that `v3` means one specific set of bytes forever, and that a model card can name it.\n\n## Why to do this even for a project nobody else will see\n\nTwo reasons, both selfish.\n\nThe first: in four months you will be the stranger. You will not remember whether the refund flattening was deliberate, and you will spend an afternoon reconstructing it from a script.\n\nThe second: writing the gaps section changes the dataset. Nearly every time somebody sits down to list what their data does not cover, they stop halfway through and go and collect some of it. That is the document doing its job before it has been read by anybody."},"check":{"question":"Why is the per-filter removal count more useful than a list of the filters that were applied?","options":[{"text":"It proves the filters ran, which a list alone cannot establish for an audit.","correct":false,"tempting":"Frames the value as evidentiary, when the counts are chiefly a diagnostic for explaining later model behaviour."},{"text":"It shows whether a filter removed a surprising share, and points at the cause of a later failure.","correct":true},{"text":"It lets you recompute the dataset without rerunning the filters, by subtracting the counts from the source totals.","correct":false,"tempting":"Treats counts as a substitute for the data itself; knowing 887 rows were removed does not tell you which ones."},{"text":"It allows the filters to be reordered optimally, since cheaper filters with high removal rates should run first.","correct":false,"tempting":"Describes a genuine efficiency consideration that has nothing to do with why the counts belong in a document read months later."}]},"prev":{"slug":"privacy-and-memorisation","title":"Memorisation, and what you cannot take back","url":"https://addaly.com/learn/fine-tuning/privacy-and-memorisation"},"next":{"slug":"sft-is-imitation","title":"What supervised fine-tuning cannot express","url":"https://addaly.com/learn/fine-tuning/sft-is-imitation"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The data card', in Fine-Tuning, and When Not To. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/fine-tuning/documenting-the-dataset (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The data card","courseTitle":"Fine-Tuning, and When Not To","module":{"index":4,"slug":"building-the-dataset","title":"Building the dataset"},"answer":null,"keyIdea":"A data card with per-filter counts, an honest known-gaps section, a content hash and a one-command rebuild is what links a model to the data that made it, and writing the gaps section usually changes the dataset before anybody else reads it.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-artefact-that-outlives-the-model","runs":[{"text":"The artefact that outlives the model"}]},{"t":"p","runs":[{"text":"Models get replaced. Datasets get reused, extended and argued about for years. The document that makes that possible is a data card, and it takes forty minutes to write."}]},{"t":"p","runs":[{"text":"The practice comes from "},{"text":"Datasheets for Datasets","i":true},{"text":", which proposed that every dataset ship with a record of its motivation, composition, collection process and recommended uses: the way an electronic component ships with a datasheet. The full questionnaire is long; the version below is the part that earns its keep for an applied fine-tune."}]},{"t":"h2","id":"the-template","runs":[{"text":"The template"}]},{"t":"code","lang":"markdown","label":"Markdown","text":"# Dataset: ticket-triage-sft, v3\n\n## What it is\n31,084 single-turn examples mapping a customer message to a triage\ndecision and a short reply. English only.\n\n## Sources and counts\n- support_logs   24,003  internal, ToS v4 analytics basis, 2026-01 to 2026-08\n- hand_written    2,140  written by the support team against style-guide v2\n- oasst1          4,941  Apache-2.0, replay mixture (16% of final)\n\n## Filters applied, with counts\nstart 48,201 → exact dup -6,340 → minhash 0.85 -9,118 → short -412\n→ truncated -887 → non-English -204 → repetition -156 → final 31,084\n\n## Processing\n- Presidio redaction (emails, phones, order refs, names) before storage\n- 200 rows manually reviewed post-redaction; 3 leaks found and fixed\n- Converted to messages format by scripts/to_messages.py @ 4f1a09\n\n## Known gaps\n- No examples of the model declining an out-of-scope request other than\n  the 88 hand-written ones. Under-represented.\n- Refunds are 11% of the set and 1% of traffic (deliberate, flattened).\n- Nothing after 2026-08; the September pricing change is not represented.\n- No non-English, though ~4% of traffic is.\n\n## Licence and constraints\nInternal use only. Contains the oasst1 subset (Apache-2.0, attribution\nkept in source column). Not redistributable.\n\n## Reproducing it\nmake dataset   # from raw/, deterministic given seed 0\nsha256(train.jsonl) = 9f2c1e...\n\n## Owner\nr.kapoor — questions about labelling decisions go here"},{"t":"h2","id":"the-four-sections-that-do-the-work","runs":[{"text":"The four sections that do the work"}]},{"t":"p","runs":[{"text":"Filters with counts.","b":true},{"text":" When somebody later asks why the model is bad at long tickets, the line "},{"text":"truncated -887","code":true},{"text":" is the first place to look. A filter list without counts is decoration."}]},{"t":"p","runs":[{"text":"Known gaps.","b":true},{"text":" The hardest section to write and the most valuable. It requires admitting what the dataset does not cover, which is exactly the information the next person needs and exactly what an enthusiastic project write-up omits. Every production surprise I have seen traced back to something that would have been in a properly written gaps section."}]},{"t":"p","runs":[{"text":"The hash.","b":true},{"text":" One line, and it is what connects a model to the data that made it. Without it, “which dataset produced this checkpoint” is answered from memory. Put the hash in the model’s own metadata too, so the link survives in both directions."}]},{"t":"p","runs":[{"text":"Reproducing it.","b":true},{"text":" If the answer is a notebook on somebody’s laptop, write that down honestly, because that is a risk the organisation should know it is carrying. If the answer is one command, you have a dataset that survives its author leaving."}]},{"t":"h2","id":"version-it-like-code","runs":[{"text":"Version it like code"}]},{"t":"p","runs":[{"text":"The dataset is an input to the model, so it belongs in version control with the same discipline. Small enough files go in git directly; larger ones go in git-lfs, DVC, or a Hugging Face dataset repository, all of which are free and all of which give you a content hash."}]},{"t":"p","runs":[{"text":"What matters is that "},{"text":"v3","code":true},{"text":" means one specific set of bytes forever, and that a model card can name it."}]},{"t":"h2","id":"why-to-do-this-even-for-a-project-nobody-else-will-see","runs":[{"text":"Why to do this even for a project nobody else will see"}]},{"t":"p","runs":[{"text":"Two reasons, both selfish."}]},{"t":"p","runs":[{"text":"The first: in four months you will be the stranger. You will not remember whether the refund flattening was deliberate, and you will spend an afternoon reconstructing it from a script."}]},{"t":"p","runs":[{"text":"The second: writing the gaps section changes the dataset. Nearly every time somebody sits down to list what their data does not cover, they stop halfway through and go and collect some of it. That is the document doing its job before it has been read by anybody."}]}],"check":{"question":[{"text":"Why is the per-filter removal count more useful than a list of the filters that were applied?"}],"options":[{"text":[{"text":"It proves the filters ran, which a list alone cannot establish for an audit."}],"correct":false,"tempting":[{"text":"Frames the value as evidentiary, when the counts are chiefly a diagnostic for explaining later model behaviour."}]},{"text":[{"text":"It shows whether a filter removed a surprising share, and points at the cause of a later failure."}],"correct":true,"tempting":null},{"text":[{"text":"It lets you recompute the dataset without rerunning the filters, by subtracting the counts from the source totals."}],"correct":false,"tempting":[{"text":"Treats counts as a substitute for the data itself; knowing 887 rows were removed does not tell you which ones."}]},{"text":[{"text":"It allows the filters to be reordered optimally, since cheaper filters with high removal rates should run first."}],"correct":false,"tempting":[{"text":"Describes a genuine efficiency consideration that has nothing to do with why the counts belong in a document read months later."}]}],"order":[3,1,2,0],"why":null},"prev":{"slug":"privacy-and-memorisation","title":"Memorisation, and what you cannot take back","minutes":10},"next":{"slug":"sft-is-imitation","title":"What supervised fine-tuning cannot express","minutes":9}}}