{"course":{"slug":"fine-tuning","title":"Fine-Tuning, and When Not To","url":"https://addaly.com/learn/fine-tuning","level":"advanced","track":"deep"},"lesson":{"slug":"versioning-models-and-data","title":"Versioning so a model can be explained later","url":"https://addaly.com/learn/fine-tuning/versioning-models-and-data","position":82,"of":84,"minutes":8,"answer":"A manifest of content hashes travelling with the artefact, plus a model-version field on every logged request, is what turns 'what produced this answer in March' from a memory into a lookup.","keyIdea":"A manifest of content hashes travelling with the artefact, plus a model-version field on every logged request, is what turns 'what produced this answer in March' from a memory into a lookup; and rebuilding a random old version once a quarter is how you find out whether it actually works."},"body":{"format":"markdown","text":"## The question you will be asked\n\nSomeone forwards a complaint about a decision the system made in March. What produced that answer?\n\nAnswering it needs six things linked together: the request, the model version that served it, the adapter artefact, the base model revision, the dataset that trained it, and the configuration. Miss any one and the chain breaks.\n\n## The manifest\n\nOne file, written at build time, travelling with the artefact:\n\n```json\n{\"name\": \"triage\", \"version\": \"3.2.0\", \"built\": \"2026-09-15T11:04Z\",\n \"base_model\": \"meta-llama/Llama-3.1-8B-Instruct\",\n \"base_revision\": \"0e9e39f31a2fc8d\",\n \"dataset\": \"ticket-triage-sft@v3\",\n \"dataset_sha256\": \"9f2c1e…\",\n \"config_sha256\": \"71ba04…\",\n \"eval_set\": \"eval-v3\", \"eval_sha256\": \"c31d77…\",\n \"git_commit\": \"4f1a09\",\n \"results\": {\"task\": 0.918, \"canary\": 0.812, \"refusal\": 0.94},\n \"quantisation\": \"Q4_K_M\",\n \"seed\": 0}\n```\n\nEverything in it is a hash or an identifier that resolves to exact bytes. \"We trained on the September dataset\" is a memory; `9f2c1e…` is a fact.\n\n## Log the version on every request\n\n```json\n{\"ts\": \"2026-03-04T09:12Z\", \"request_id\": \"r_88412\",\n \"model\": \"triage@3.2.0\", \"adapter\": \"triage-3.2.0\", \"latency_ms\": 340}\n```\n\nThis single field is what connects a complaint to an artefact. Without it, a March request maps to \"whatever was deployed in March\", which nobody is certain about after two rollbacks.\n\n## Version numbers that carry meaning\n\nA convention worth adopting:\n\n- **Major**: the base model changed. Behaviour may differ everywhere; re-evaluate everything.\n- **Minor**: the dataset changed. Same base, new training data.\n- **Patch**: the same weights, a different build: a new quantisation, a new serving format.\n\nSo 3.2.0 and 3.2.1 are the same model in different formats, and their evaluation numbers will differ slightly, which is exactly the sort of thing that looks like a mystery until the version scheme tells you what changed.\n\n## Where the bytes live\n\nAll free:\n\n- **Code and configs**: git.\n- **Datasets**: a Hugging Face dataset repository (private repositories are free), or DVC or git-lfs against your own storage. All three give content-addressed versions.\n- **Model artefacts**: a Hub model repository, or object storage with the version in the path.\n- **Run metadata and results**: MLflow's registry if you want a searchable index, or a JSON file per build if you do not.\n\nA fully adequate small-team setup is: code in git, dataset and adapter in private Hub repositories, manifest committed to git beside the config. No infrastructure, no cost, and it answers the March question.\n\n## Retention\n\nKeep enough to explain a decision for as long as you might be asked about it. That period is set by your product and your obligations, not by disk cost; and a LoRA adapter is a few hundred megabytes, so the disk cost is not the constraint.\n\nWhat to keep per version: the manifest, the adapter, the evaluation results, and a pointer to the dataset version. What you do not need to keep: every intermediate checkpoint, and the optimiser states, which are large and only useful for resuming a run you will not resume.\n\n## Three things that break this in practice\n\nEvery versioning scheme that fails does so in one of three ways, and all three are cheap to prevent.\n\n**A configuration value that is not in the hash.** The config file is hashed, but the batch size comes from an environment variable, or the base model path from a shell default. Two runs with identical config hashes then differ. The fix is to resolve every value into the config file before hashing it, and to have the training script write out the *resolved* config as an artefact rather than the template.\n\n**A dataset built by something that is not deterministic.** A notebook cell that samples without a seed, a query with `LIMIT 20000` and no `ORDER BY`, a filter that reads the current date. The hash then records what you built, which is correct and useless, because rerunning the builder produces a different file. Seed everything in the pipeline and pin the query, or accept that the dataset is an artefact to be stored rather than a recipe to be rerun: both work, and pretending you have the second when you have the first does not.\n\n**A floating reference to the base model.** `main`, `latest`, or a tag that was moved. The manifest looks complete and resolves to different bytes over time. Always record the commit hash of the model repository, not its name.\n\n## Promotion, not just versioning\n\nA version number says what something is. A *stage* says where it is allowed to run.\n\n```\ntriage@3.2.0   stage: production   since 2026-09-15\ntriage@3.3.0   stage: staging      shadowing production traffic\ntriage@3.4.0   stage: development\n```\n\nThree stages is enough. The value is that \"which model is in production\" becomes a fact stored somewhere rather than an inference from deployment logs, and rolling back becomes moving a pointer rather than rebuilding something.\n\n## The test\n\nOnce a quarter, pick a version at random and rebuild it from its manifest. Compare its evaluation numbers against the recorded ones.\n\nIf they match within noise, your versioning works. If you cannot rebuild it at all, you have discovered that on a Tuesday of your choosing rather than during an incident, which is the entire value of the exercise."},"check":{"question":"A team stores adapters in object storage with a date in the filename and records the dataset as 'the September export'. What specifically fails when they are asked to explain a decision made six months earlier?","options":[{"text":"Object storage does not preserve file integrity over six months, so the adapter may have been altered.","correct":false,"tempting":"Invents a storage-durability problem; object stores preserve bytes, and the gap is in knowing which bytes were used."},{"text":"Nothing links a request to an adapter, and 'the September export' resolves to no exact bytes.","correct":true},{"text":"The date-based filenames make it impossible to tell which adapter is newest, so the wrong one may be examined.","correct":false,"tempting":"Dates order artefacts adequately; the failure is not in ordering them but in linking one to a request and to its inputs."},{"text":"Without a model registry, the adapter cannot be loaded by the serving stack after a framework upgrade.","correct":false,"tempting":"Raises a real format-compatibility risk that a registry does not solve either, and that is separate from identifying what ran."}]},"prev":{"slug":"the-base-model-moved","title":"The day the base model moves","url":"https://addaly.com/learn/fine-tuning/the-base-model-moved"},"next":{"slug":"disclosure-and-model-cards","title":"Model cards and disclosure","url":"https://addaly.com/learn/fine-tuning/disclosure-and-model-cards"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Versioning so a model can be explained later', in Fine-Tuning, and When Not To. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/fine-tuning/versioning-models-and-data (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Versioning so a model can be explained later","courseTitle":"Fine-Tuning, and When Not To","module":{"index":9,"slug":"shipping-and-living-with-it","title":"Shipping it, and living with it"},"answer":null,"keyIdea":"A manifest of content hashes travelling with the artefact, plus a model-version field on every logged request, is what turns ‘what produced this answer in March’ from a memory into a lookup; and rebuilding a random old version once a quarter is how you find out whether it actually works.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-question-you-will-be-asked","runs":[{"text":"The question you will be asked"}]},{"t":"p","runs":[{"text":"Someone forwards a complaint about a decision the system made in March. What produced that answer?"}]},{"t":"p","runs":[{"text":"Answering it needs six things linked together: the request, the model version that served it, the adapter artefact, the base model revision, the dataset that trained it, and the configuration. Miss any one and the chain breaks."}]},{"t":"h2","id":"the-manifest","runs":[{"text":"The manifest"}]},{"t":"p","runs":[{"text":"One file, written at build time, travelling with the artefact:"}]},{"t":"code","lang":"json","label":"JSON","text":"{\"name\": \"triage\", \"version\": \"3.2.0\", \"built\": \"2026-09-15T11:04Z\",\n \"base_model\": \"meta-llama/Llama-3.1-8B-Instruct\",\n \"base_revision\": \"0e9e39f31a2fc8d\",\n \"dataset\": \"ticket-triage-sft@v3\",\n \"dataset_sha256\": \"9f2c1e…\",\n \"config_sha256\": \"71ba04…\",\n \"eval_set\": \"eval-v3\", \"eval_sha256\": \"c31d77…\",\n \"git_commit\": \"4f1a09\",\n \"results\": {\"task\": 0.918, \"canary\": 0.812, \"refusal\": 0.94},\n \"quantisation\": \"Q4_K_M\",\n \"seed\": 0}"},{"t":"p","runs":[{"text":"Everything in it is a hash or an identifier that resolves to exact bytes. “We trained on the September dataset” is a memory; "},{"text":"9f2c1e…","code":true},{"text":" is a fact."}]},{"t":"h2","id":"log-the-version-on-every-request","runs":[{"text":"Log the version on every request"}]},{"t":"code","lang":"json","label":"JSON","text":"{\"ts\": \"2026-03-04T09:12Z\", \"request_id\": \"r_88412\",\n \"model\": \"triage@3.2.0\", \"adapter\": \"triage-3.2.0\", \"latency_ms\": 340}"},{"t":"p","runs":[{"text":"This single field is what connects a complaint to an artefact. Without it, a March request maps to “whatever was deployed in March”, which nobody is certain about after two rollbacks."}]},{"t":"h2","id":"version-numbers-that-carry-meaning","runs":[{"text":"Version numbers that carry meaning"}]},{"t":"p","runs":[{"text":"A convention worth adopting:"}]},{"t":"ul","items":[[{"text":"Major","b":true},{"text":": the base model changed. Behaviour may differ everywhere; re-evaluate everything."}],[{"text":"Minor","b":true},{"text":": the dataset changed. Same base, new training data."}],[{"text":"Patch","b":true},{"text":": the same weights, a different build: a new quantisation, a new serving format."}]]},{"t":"p","runs":[{"text":"So 3.2.0 and 3.2.1 are the same model in different formats, and their evaluation numbers will differ slightly, which is exactly the sort of thing that looks like a mystery until the version scheme tells you what changed."}]},{"t":"h2","id":"where-the-bytes-live","runs":[{"text":"Where the bytes live"}]},{"t":"p","runs":[{"text":"All free:"}]},{"t":"ul","items":[[{"text":"Code and configs","b":true},{"text":": git."}],[{"text":"Datasets","b":true},{"text":": a Hugging Face dataset repository (private repositories are free), or DVC or git-lfs against your own storage. All three give content-addressed versions."}],[{"text":"Model artefacts","b":true},{"text":": a Hub model repository, or object storage with the version in the path."}],[{"text":"Run metadata and results","b":true},{"text":": MLflow’s registry if you want a searchable index, or a JSON file per build if you do not."}]]},{"t":"p","runs":[{"text":"A fully adequate small-team setup is: code in git, dataset and adapter in private Hub repositories, manifest committed to git beside the config. No infrastructure, no cost, and it answers the March question."}]},{"t":"h2","id":"retention","runs":[{"text":"Retention"}]},{"t":"p","runs":[{"text":"Keep enough to explain a decision for as long as you might be asked about it. That period is set by your product and your obligations, not by disk cost; and a LoRA adapter is a few hundred megabytes, so the disk cost is not the constraint."}]},{"t":"p","runs":[{"text":"What to keep per version: the manifest, the adapter, the evaluation results, and a pointer to the dataset version. What you do not need to keep: every intermediate checkpoint, and the optimiser states, which are large and only useful for resuming a run you will not resume."}]},{"t":"h2","id":"three-things-that-break-this-in-practice","runs":[{"text":"Three things that break this in practice"}]},{"t":"p","runs":[{"text":"Every versioning scheme that fails does so in one of three ways, and all three are cheap to prevent."}]},{"t":"p","runs":[{"text":"A configuration value that is not in the hash.","b":true},{"text":" The config file is hashed, but the batch size comes from an environment variable, or the base model path from a shell default. Two runs with identical config hashes then differ. The fix is to resolve every value into the config file before hashing it, and to have the training script write out the "},{"text":"resolved","i":true},{"text":" config as an artefact rather than the template."}]},{"t":"p","runs":[{"text":"A dataset built by something that is not deterministic.","b":true},{"text":" A notebook cell that samples without a seed, a query with "},{"text":"LIMIT 20000","code":true},{"text":" and no "},{"text":"ORDER BY","code":true},{"text":", a filter that reads the current date. The hash then records what you built, which is correct and useless, because rerunning the builder produces a different file. Seed everything in the pipeline and pin the query, or accept that the dataset is an artefact to be stored rather than a recipe to be rerun: both work, and pretending you have the second when you have the first does not."}]},{"t":"p","runs":[{"text":"A floating reference to the base model.","b":true},{"text":" "},{"text":"main","code":true},{"text":", "},{"text":"latest","code":true},{"text":", or a tag that was moved. The manifest looks complete and resolves to different bytes over time. Always record the commit hash of the model repository, not its name."}]},{"t":"h2","id":"promotion-not-just-versioning","runs":[{"text":"Promotion, not just versioning"}]},{"t":"p","runs":[{"text":"A version number says what something is. A "},{"text":"stage","i":true},{"text":" says where it is allowed to run."}]},{"t":"code","lang":"","label":"Text","text":"triage@3.2.0   stage: production   since 2026-09-15\ntriage@3.3.0   stage: staging      shadowing production traffic\ntriage@3.4.0   stage: development"},{"t":"p","runs":[{"text":"Three stages is enough. The value is that “which model is in production” becomes a fact stored somewhere rather than an inference from deployment logs, and rolling back becomes moving a pointer rather than rebuilding something."}]},{"t":"h2","id":"the-test","runs":[{"text":"The test"}]},{"t":"p","runs":[{"text":"Once a quarter, pick a version at random and rebuild it from its manifest. Compare its evaluation numbers against the recorded ones."}]},{"t":"p","runs":[{"text":"If they match within noise, your versioning works. If you cannot rebuild it at all, you have discovered that on a Tuesday of your choosing rather than during an incident, which is the entire value of the exercise."}]}],"check":{"question":[{"text":"A team stores adapters in object storage with a date in the filename and records the dataset as ‘the September export’. What specifically fails when they are asked to explain a decision made six months earlier?"}],"options":[{"text":[{"text":"Object storage does not preserve file integrity over six months, so the adapter may have been altered."}],"correct":false,"tempting":[{"text":"Invents a storage-durability problem; object stores preserve bytes, and the gap is in knowing which bytes were used."}]},{"text":[{"text":"Nothing links a request to an adapter, and ‘the September export’ resolves to no exact bytes."}],"correct":true,"tempting":null},{"text":[{"text":"The date-based filenames make it impossible to tell which adapter is newest, so the wrong one may be examined."}],"correct":false,"tempting":[{"text":"Dates order artefacts adequately; the failure is not in ordering them but in linking one to a request and to its inputs."}]},{"text":[{"text":"Without a model registry, the adapter cannot be loaded by the serving stack after a framework upgrade."}],"correct":false,"tempting":[{"text":"Raises a real format-compatibility risk that a registry does not solve either, and that is separate from identifying what ran."}]}],"order":[3,2,1,0],"why":null},"prev":{"slug":"the-base-model-moved","title":"The day the base model moves","minutes":9},"next":{"slug":"disclosure-and-model-cards","title":"Model cards and disclosure","minutes":9}}}