{"course":{"slug":"running-models-yourself","title":"Running Models Yourself","url":"https://addaly.com/learn/running-models-yourself","level":"intermediate","track":"deep"},"lesson":{"slug":"building-the-dataset","title":"The dataset is the project","url":"https://addaly.com/learn/running-models-yourself/building-the-dataset","position":77,"of":82,"minutes":9,"answer":"Render your training examples with the model's own chat template and read the output before training, compute loss on assistant tokens only.","keyIdea":"Render your training examples with the model's own chat template and read the output before training, compute loss on assistant tokens only, include refusals and awkward shapes because a dataset of successes teaches a model to always succeed, and record provenance per example while it is still possible."},"body":{"format":"markdown","text":"## Five hundred good examples beat fifty thousand scraped ones\n\nEvery decision in a fine-tuning project is downstream of the data, and the amount of data people imagine they need is usually an order of magnitude more than they do. For a behaviour, a format, or a classification boundary, several hundred carefully chosen examples routinely outperform tens of thousands of noisy ones, because the noisy set teaches the noise too, and the model has no way to tell which parts you meant.\n\nSpend the effort here. A week on the dataset and an afternoon on hyperparameters is the correct ratio, and it is almost always inverted by people doing this for the first time.\n\n## Use the template you will serve with\n\nThe most common silent failure in fine-tuning is a training format that does not match the inference format. If you train on plain text like `Q: ... A: ...` and then serve through a chat endpoint that wraps messages in the model's own special tokens, the model at inference time is seeing a structure it never trained on. Behaviour is degraded, nothing errors, and the cause is invisible.\n\nStore examples as messages and render them with the model's own template:\n\n```python\nfrom transformers import AutoTokenizer\ntok = AutoTokenizer.from_pretrained(MODEL)\nrow = {\"messages\": [\n  {\"role\": \"system\", \"content\": \"You are a triage assistant for a clinic.\"},\n  {\"role\": \"user\", \"content\": \"Appointment moved to Friday, please confirm.\"},\n  {\"role\": \"assistant\", \"content\": \"{\\\"intent\\\":\\\"reschedule\\\",\\\"urgency\\\":\\\"low\\\"}\"},\n]}\nprint(tok.apply_chat_template(row[\"messages\"], tokenize=False))\n```\n\nPrint that rendered string and read it before you train anything. Ten seconds of reading catches a missing system turn, a doubled beginning-of-sequence token, or a template that puts the assistant's reply somewhere you did not expect.\n\n## Mask the prompt, train on the response\n\nBy default, a language model trains on every token in the sequence, including the user's message. That teaches it to generate user turns, which is not what you want, and it dilutes the gradient with tokens whose content you are not trying to change.\n\nInstruction tuning should compute loss on the assistant's tokens only. Most frameworks expose this directly; in TRL it is a completion-only collator, and Unsloth ships a `train_on_responses_only` helper. It is one line and it reliably improves the result.\n\nThe exception is worth knowing: if you are teaching a completion behaviour rather than a chat behaviour, training on the whole sequence is correct. The rule is that loss should be computed on exactly the tokens the model will be asked to produce.\n\n## What to put in, beyond the happy path\n\nA dataset of successes teaches a model to always succeed, including when it should not.\n\n- **Include refusals and uncertainty.** If you never show it an input it should decline or ask about, it will confidently handle those too. A tenth of the set being \"I do not have enough information to do this\" changes the model's behaviour on ambiguous inputs markedly.\n- **Include the awkward shapes.** Empty fields, the wrong language, a message that is three words long, a message that is four pages. Whatever your real traffic contains.\n- **Watch the length distribution.** Output length is learned. If every training response is two sentences, you will get two sentences forever, including when the user asks for a detailed explanation. Vary it deliberately to match what you want.\n- **Balance the classes you care about.** A rare label at 2% of the data will stay rare in the model's behaviour. Oversample it, or accept that it will be under-predicted.\n\n## Split before you look\n\nHold out 10 to 20% before you start, stratified by the input shapes and the labels that matter, and do not look at it. Every decision you make after seeing the held-out set leaks into it, and the number it gives you afterwards is no longer a measurement.\n\nKeep a third, smaller set that is not from the same source at all (collected a month later, or from a different team) because a train-and-test split from one collection shares its quirks, and a model that looks excellent on both can still fail on real traffic for reasons the collection never contained.\n\n## Provenance, recorded as you go\n\nWrite down where every example came from, at the time, in the file. It takes seconds then and is impossible to reconstruct later.\n\nThis matters practically and legally. Data scraped from a site carries that site's terms. Data from a public dataset carries its licence, which is frequently more restrictive than people assume. Output generated by a hosted model carries that provider's terms of use, and several providers restrict using their output to develop competing models. Whether a narrow task adapter counts as a competing model is contested and largely untested. Personal data in a training set raises its own obligations, complicated by the fact that removing an example from a dataset does not remove what a model learned from it.\n\nThis course cannot tell you your legal position and is not legal advice. What it can tell you is that a provenance column costs nothing to keep and is the only thing that lets anyone answer these questions later."},"check":{"question":"A fine-tuned support model handles clear requests well but confidently invents details whenever a message is ambiguous, where the base model used to ask a clarifying question. What is the most likely cause in the dataset?","options":[{"text":"The dataset was too small, so the model fell back on base behaviour for inputs outside its coverage.","correct":false,"tempting":"Describes the opposite of what happened: the base behaviour was to ask for clarification, and that is precisely what the tuning removed."},{"text":"Loss was computed over the whole sequence rather than assistant tokens only, so the model learned to generate user turns.","correct":false,"tempting":"A genuine error with a different signature (spurious user-style text in outputs) rather than confident answers to ambiguous inputs."},{"text":"Every example showed a confident answer, so declining was never demonstrated as acceptable.","correct":true},{"text":"The learning rate was too high, so the model overwrote the base model's instruction-following behaviour.","correct":false,"tempting":"An excessive rate degrades behaviour broadly, whereas this failure is specific to exactly the input class the dataset never represented."}]},"prev":{"slug":"qlora-on-one-card","title":"QLoRA: training a 7B on the card you have","url":"https://addaly.com/learn/running-models-yourself/qlora-on-one-card"},"next":{"slug":"running-the-training-run","title":"Running the training run","url":"https://addaly.com/learn/running-models-yourself/running-the-training-run"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The dataset is the project', in Running Models Yourself. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/running-models-yourself/building-the-dataset (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The dataset is the project","courseTitle":"Running Models Yourself","module":{"index":9,"slug":"making-it-yours","title":"Making it yours: adapters and training"},"answer":null,"keyIdea":"Render your training examples with the model’s own chat template and read the output before training, compute loss on assistant tokens only, include refusals and awkward shapes because a dataset of successes teaches a model to always succeed, and record provenance per example while it is still possible.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"five-hundred-good-examples-beat-fifty-thousand-scraped-ones","runs":[{"text":"Five hundred good examples beat fifty thousand scraped ones"}]},{"t":"p","runs":[{"text":"Every decision in a fine-tuning project is downstream of the data, and the amount of data people imagine they need is usually an order of magnitude more than they do. For a behaviour, a format, or a classification boundary, several hundred carefully chosen examples routinely outperform tens of thousands of noisy ones, because the noisy set teaches the noise too, and the model has no way to tell which parts you meant."}]},{"t":"p","runs":[{"text":"Spend the effort here. A week on the dataset and an afternoon on hyperparameters is the correct ratio, and it is almost always inverted by people doing this for the first time."}]},{"t":"h2","id":"use-the-template-you-will-serve-with","runs":[{"text":"Use the template you will serve with"}]},{"t":"p","runs":[{"text":"The most common silent failure in fine-tuning is a training format that does not match the inference format. If you train on plain text like "},{"text":"Q: ... A: ...","code":true},{"text":" and then serve through a chat endpoint that wraps messages in the model’s own special tokens, the model at inference time is seeing a structure it never trained on. Behaviour is degraded, nothing errors, and the cause is invisible."}]},{"t":"p","runs":[{"text":"Store examples as messages and render them with the model’s own template:"}]},{"t":"code","lang":"python","label":"Python","text":"from transformers import AutoTokenizer\ntok = AutoTokenizer.from_pretrained(MODEL)\nrow = {\"messages\": [\n  {\"role\": \"system\", \"content\": \"You are a triage assistant for a clinic.\"},\n  {\"role\": \"user\", \"content\": \"Appointment moved to Friday, please confirm.\"},\n  {\"role\": \"assistant\", \"content\": \"{\\\"intent\\\":\\\"reschedule\\\",\\\"urgency\\\":\\\"low\\\"}\"},\n]}\nprint(tok.apply_chat_template(row[\"messages\"], tokenize=False))"},{"t":"p","runs":[{"text":"Print that rendered string and read it before you train anything. Ten seconds of reading catches a missing system turn, a doubled beginning-of-sequence token, or a template that puts the assistant’s reply somewhere you did not expect."}]},{"t":"h2","id":"mask-the-prompt-train-on-the-response","runs":[{"text":"Mask the prompt, train on the response"}]},{"t":"p","runs":[{"text":"By default, a language model trains on every token in the sequence, including the user’s message. That teaches it to generate user turns, which is not what you want, and it dilutes the gradient with tokens whose content you are not trying to change."}]},{"t":"p","runs":[{"text":"Instruction tuning should compute loss on the assistant’s tokens only. Most frameworks expose this directly; in TRL it is a completion-only collator, and Unsloth ships a "},{"text":"train_on_responses_only","code":true},{"text":" helper. It is one line and it reliably improves the result."}]},{"t":"p","runs":[{"text":"The exception is worth knowing: if you are teaching a completion behaviour rather than a chat behaviour, training on the whole sequence is correct. The rule is that loss should be computed on exactly the tokens the model will be asked to produce."}]},{"t":"h2","id":"what-to-put-in-beyond-the-happy-path","runs":[{"text":"What to put in, beyond the happy path"}]},{"t":"p","runs":[{"text":"A dataset of successes teaches a model to always succeed, including when it should not."}]},{"t":"ul","items":[[{"text":"Include refusals and uncertainty.","b":true},{"text":" If you never show it an input it should decline or ask about, it will confidently handle those too. A tenth of the set being “I do not have enough information to do this” changes the model’s behaviour on ambiguous inputs markedly."}],[{"text":"Include the awkward shapes.","b":true},{"text":" Empty fields, the wrong language, a message that is three words long, a message that is four pages. Whatever your real traffic contains."}],[{"text":"Watch the length distribution.","b":true},{"text":" Output length is learned. If every training response is two sentences, you will get two sentences forever, including when the user asks for a detailed explanation. Vary it deliberately to match what you want."}],[{"text":"Balance the classes you care about.","b":true},{"text":" A rare label at 2% of the data will stay rare in the model’s behaviour. Oversample it, or accept that it will be under-predicted."}]]},{"t":"h2","id":"split-before-you-look","runs":[{"text":"Split before you look"}]},{"t":"p","runs":[{"text":"Hold out 10 to 20% before you start, stratified by the input shapes and the labels that matter, and do not look at it. Every decision you make after seeing the held-out set leaks into it, and the number it gives you afterwards is no longer a measurement."}]},{"t":"p","runs":[{"text":"Keep a third, smaller set that is not from the same source at all (collected a month later, or from a different team) because a train-and-test split from one collection shares its quirks, and a model that looks excellent on both can still fail on real traffic for reasons the collection never contained."}]},{"t":"h2","id":"provenance-recorded-as-you-go","runs":[{"text":"Provenance, recorded as you go"}]},{"t":"p","runs":[{"text":"Write down where every example came from, at the time, in the file. It takes seconds then and is impossible to reconstruct later."}]},{"t":"p","runs":[{"text":"This matters practically and legally. Data scraped from a site carries that site’s terms. Data from a public dataset carries its licence, which is frequently more restrictive than people assume. Output generated by a hosted model carries that provider’s terms of use, and several providers restrict using their output to develop competing models. Whether a narrow task adapter counts as a competing model is contested and largely untested. Personal data in a training set raises its own obligations, complicated by the fact that removing an example from a dataset does not remove what a model learned from it."}]},{"t":"p","runs":[{"text":"This course cannot tell you your legal position and is not legal advice. What it can tell you is that a provenance column costs nothing to keep and is the only thing that lets anyone answer these questions later."}]}],"check":{"question":[{"text":"A fine-tuned support model handles clear requests well but confidently invents details whenever a message is ambiguous, where the base model used to ask a clarifying question. What is the most likely cause in the dataset?"}],"options":[{"text":[{"text":"The dataset was too small, so the model fell back on base behaviour for inputs outside its coverage."}],"correct":false,"tempting":[{"text":"Describes the opposite of what happened: the base behaviour was to ask for clarification, and that is precisely what the tuning removed."}]},{"text":[{"text":"Loss was computed over the whole sequence rather than assistant tokens only, so the model learned to generate user turns."}],"correct":false,"tempting":[{"text":"A genuine error with a different signature (spurious user-style text in outputs) rather than confident answers to ambiguous inputs."}]},{"text":[{"text":"Every example showed a confident answer, so declining was never demonstrated as acceptable."}],"correct":true,"tempting":null},{"text":[{"text":"The learning rate was too high, so the model overwrote the base model’s instruction-following behaviour."}],"correct":false,"tempting":[{"text":"An excessive rate degrades behaviour broadly, whereas this failure is specific to exactly the input class the dataset never represented."}]}],"order":[1,3,2,0],"why":null},"prev":{"slug":"qlora-on-one-card","title":"QLoRA: training a 7B on the card you have","minutes":10},"next":{"slug":"running-the-training-run","title":"Running the training run","minutes":9}}}