{"course":{"slug":"fine-tuning","title":"Fine-Tuning, and When Not To","url":"https://addaly.com/learn/fine-tuning","level":"advanced","track":"deep"},"lesson":{"slug":"disclosure-and-model-cards","title":"Model cards and disclosure","url":"https://addaly.com/learn/fine-tuning/disclosure-and-model-cards","position":83,"of":84,"minutes":9,"answer":"A model card states intended use, out-of-scope use, training-data provenance, evaluation with intervals, and limitations explained by mechanism, and since the regulatory position differs by jurisdiction and is still moving.","keyIdea":"A model card states intended use, out-of-scope use, training-data provenance, evaluation with intervals, and limitations explained by mechanism, and since the regulatory position differs by jurisdiction and is still moving, that document is what travels everywhere while the specific legal duties do not."},"body":{"format":"markdown","text":"## The document that answers the questions\n\nA model card is to a model what a data card is to a dataset: the page a stranger reads to decide whether they can rely on it. It is also, in practice, the page that answers procurement questionnaires, security reviews and the first hour of an incident.\n\nIt takes an hour to write and it is written once per major version.\n\n```markdown\n# triage-3.2.0\n\n## What it does\nRoutes a customer message to one of six queues and drafts a one-paragraph\nreply. English only.\n\n## Base and licence\nLlama-3.1-8B-Instruct @ 0e9e39f, under the Llama 3.1 Community Licence.\nNaming and acceptable-use terms follow this derivative.\nAdapter: LoRA r=16, all-linear. Merged and quantised to Q4_K_M.\n\n## Training data\nticket-triage-sft@v3 — 31,084 examples: 24,003 internal support logs\n(ToS v4 basis, PII-redacted), 2,140 hand-written, 4,941 OpenAssistant\n(Apache-2.0) as replay. Data card: docs/data/ticket-triage-v3.md\n\n## Evaluation\nHeld-out n=412, drawn after the training cutoff, group-split by customer.\n  routing accuracy   0.918  [0.888 – 0.941]\n  valid JSON         1.000  (constrained decoding)\n  canary (general)   0.812  (base 0.840)\n  refusal set        0.94   (base 0.96)\nSeed noise floor: sd 1.3 points over 3 runs.\nJudged by: exact match on queue; replies by human pairwise, n=200.\n\n## Intended use\nFirst-line triage with a human reviewing anything the model routes to\n'refunds' or 'escalation'.\n\n## Out of scope\nNot for medical, legal or financial advice. Not for non-English input.\nNot for deciding refunds. Not an identity or eligibility check.\n\n## Known limitations\n- No data after 2026-08; the September pricing change is not represented.\n- 4% of live traffic is non-English and is routed to 'other' by design.\n- Canary is 2.8 points below base: general capability is slightly reduced.\n- Confidence is uncalibrated above 0.9; do not threshold there.\n\n## Contact\nr.kapoor\n```\n\n## The sections that earn their space\n\n**Out of scope.** The most useful section and the hardest to write, because it requires saying no to uses somebody has already imagined. It is also the section that protects you, because a stated boundary is a decision rather than an oversight.\n\n**Known limitations, with mechanisms.** Not \"may occasionally be inaccurate\" (which is true of everything and informs nobody) but the specific, mechanical limits: the data cutoff, the languages, the canary gap, the calibration range.\n\n**Evaluation with intervals and provenance.** A number without an interval and without a statement of what judged it is not a result.\n\n## Disclosure to users\n\nSeparate from the card, and simpler: if a person is interacting with a model, say so, plainly, where they are looking. Do not let a name, an avatar or a writing style imply a person.\n\nThis is the right thing regardless of what any regulation requires, and it is also the minimum several regulations are converging on.\n\n## The regulatory shape, without advice\n\nThe landscape differs sharply by jurisdiction and is moving, so what follows is the shape of it rather than a statement of what you must do.\n\n- The **European Union's AI Act** takes a risk-tiered approach with transparency obligations (disclosing that a person is interacting with an AI system, marking synthetic content) plus documentation duties on providers of general-purpose models, phasing in over several years. How its obligations fall on somebody who fine-tunes a third party's model is a question with guidance still being worked out.\n- **Sectoral regulators** in many countries (financial, medical, employment) have applied existing rules to automated decisions without new AI-specific legislation, and those rules can be stricter than any AI act.\n- Several jurisdictions have **synthetic-media and labelling rules** at various stages, and they do not agree with each other.\n- Many countries have **nothing specific at all**, where general consumer-protection, data-protection and anti-discrimination law applies as it always did.\n\nSo: this is unsettled, it depends on where you operate and what the system decides, and I am not in a position to tell you what applies to you. What travels everywhere is the documentation itself. A card with intended use, out-of-scope uses, training-data provenance, evaluation results and stated limitations is what any of these regimes asks for in some form, and it is what you would want to have written regardless.\n\n## Inherited terms\n\nOne practical item people miss: your base model's licence terms generally follow into your derivative. Community licences may carry naming requirements for derivative models and acceptable-use policies that bind your downstream users. Read them once, write the consequences into the card, and the question is answered permanently."},"check":{"question":"Which limitations entry is actually useful to somebody deciding whether to rely on the model?","options":[{"text":"\"This model may occasionally produce inaccurate or inappropriate output and should be used with human oversight.\"","correct":false,"tempting":"Is true of every model ever released, so it distinguishes nothing and supports no decision."},{"text":"\"The model was evaluated on a held-out set and performs well on the intended task.\"","correct":false,"tempting":"Restates the evaluation section as a limitation, which tells a reader nothing about where the model fails."},{"text":"\"Training data ends 2026-08; confidence above 0.9 is uncalibrated and must not be thresholded.\"","correct":true},{"text":"\"Performance may vary depending on input quality, domain and phrasing, as with all language models.\"","correct":false,"tempting":"Gestures at real sensitivities without naming any of them specifically enough to act on."}]},"prev":{"slug":"versioning-models-and-data","title":"Versioning so a model can be explained later","url":"https://addaly.com/learn/fine-tuning/versioning-models-and-data"},"next":{"slug":"retiring-a-fine-tune","title":"Retiring a fine-tune","url":"https://addaly.com/learn/fine-tuning/retiring-a-fine-tune"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Model cards and disclosure', in Fine-Tuning, and When Not To. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/fine-tuning/disclosure-and-model-cards (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Model cards and disclosure","courseTitle":"Fine-Tuning, and When Not To","module":{"index":9,"slug":"shipping-and-living-with-it","title":"Shipping it, and living with it"},"answer":null,"keyIdea":"A model card states intended use, out-of-scope use, training-data provenance, evaluation with intervals, and limitations explained by mechanism, and since the regulatory position differs by jurisdiction and is still moving, that document is what travels everywhere while the specific legal duties do not.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-document-that-answers-the-questions","runs":[{"text":"The document that answers the questions"}]},{"t":"p","runs":[{"text":"A model card is to a model what a data card is to a dataset: the page a stranger reads to decide whether they can rely on it. It is also, in practice, the page that answers procurement questionnaires, security reviews and the first hour of an incident."}]},{"t":"p","runs":[{"text":"It takes an hour to write and it is written once per major version."}]},{"t":"code","lang":"markdown","label":"Markdown","text":"# triage-3.2.0\n\n## What it does\nRoutes a customer message to one of six queues and drafts a one-paragraph\nreply. English only.\n\n## Base and licence\nLlama-3.1-8B-Instruct @ 0e9e39f, under the Llama 3.1 Community Licence.\nNaming and acceptable-use terms follow this derivative.\nAdapter: LoRA r=16, all-linear. Merged and quantised to Q4_K_M.\n\n## Training data\nticket-triage-sft@v3 — 31,084 examples: 24,003 internal support logs\n(ToS v4 basis, PII-redacted), 2,140 hand-written, 4,941 OpenAssistant\n(Apache-2.0) as replay. Data card: docs/data/ticket-triage-v3.md\n\n## Evaluation\nHeld-out n=412, drawn after the training cutoff, group-split by customer.\n  routing accuracy   0.918  [0.888 – 0.941]\n  valid JSON         1.000  (constrained decoding)\n  canary (general)   0.812  (base 0.840)\n  refusal set        0.94   (base 0.96)\nSeed noise floor: sd 1.3 points over 3 runs.\nJudged by: exact match on queue; replies by human pairwise, n=200.\n\n## Intended use\nFirst-line triage with a human reviewing anything the model routes to\n'refunds' or 'escalation'.\n\n## Out of scope\nNot for medical, legal or financial advice. Not for non-English input.\nNot for deciding refunds. Not an identity or eligibility check.\n\n## Known limitations\n- No data after 2026-08; the September pricing change is not represented.\n- 4% of live traffic is non-English and is routed to 'other' by design.\n- Canary is 2.8 points below base: general capability is slightly reduced.\n- Confidence is uncalibrated above 0.9; do not threshold there.\n\n## Contact\nr.kapoor"},{"t":"h2","id":"the-sections-that-earn-their-space","runs":[{"text":"The sections that earn their space"}]},{"t":"p","runs":[{"text":"Out of scope.","b":true},{"text":" The most useful section and the hardest to write, because it requires saying no to uses somebody has already imagined. It is also the section that protects you, because a stated boundary is a decision rather than an oversight."}]},{"t":"p","runs":[{"text":"Known limitations, with mechanisms.","b":true},{"text":" Not “may occasionally be inaccurate” (which is true of everything and informs nobody) but the specific, mechanical limits: the data cutoff, the languages, the canary gap, the calibration range."}]},{"t":"p","runs":[{"text":"Evaluation with intervals and provenance.","b":true},{"text":" A number without an interval and without a statement of what judged it is not a result."}]},{"t":"h2","id":"disclosure-to-users","runs":[{"text":"Disclosure to users"}]},{"t":"p","runs":[{"text":"Separate from the card, and simpler: if a person is interacting with a model, say so, plainly, where they are looking. Do not let a name, an avatar or a writing style imply a person."}]},{"t":"p","runs":[{"text":"This is the right thing regardless of what any regulation requires, and it is also the minimum several regulations are converging on."}]},{"t":"h2","id":"the-regulatory-shape-without-advice","runs":[{"text":"The regulatory shape, without advice"}]},{"t":"p","runs":[{"text":"The landscape differs sharply by jurisdiction and is moving, so what follows is the shape of it rather than a statement of what you must do."}]},{"t":"ul","items":[[{"text":"The "},{"text":"European Union’s AI Act","b":true},{"text":" takes a risk-tiered approach with transparency obligations (disclosing that a person is interacting with an AI system, marking synthetic content) plus documentation duties on providers of general-purpose models, phasing in over several years. How its obligations fall on somebody who fine-tunes a third party’s model is a question with guidance still being worked out."}],[{"text":"Sectoral regulators","b":true},{"text":" in many countries (financial, medical, employment) have applied existing rules to automated decisions without new AI-specific legislation, and those rules can be stricter than any AI act."}],[{"text":"Several jurisdictions have "},{"text":"synthetic-media and labelling rules","b":true},{"text":" at various stages, and they do not agree with each other."}],[{"text":"Many countries have "},{"text":"nothing specific at all","b":true},{"text":", where general consumer-protection, data-protection and anti-discrimination law applies as it always did."}]]},{"t":"p","runs":[{"text":"So: this is unsettled, it depends on where you operate and what the system decides, and I am not in a position to tell you what applies to you. What travels everywhere is the documentation itself. A card with intended use, out-of-scope uses, training-data provenance, evaluation results and stated limitations is what any of these regimes asks for in some form, and it is what you would want to have written regardless."}]},{"t":"h2","id":"inherited-terms","runs":[{"text":"Inherited terms"}]},{"t":"p","runs":[{"text":"One practical item people miss: your base model’s licence terms generally follow into your derivative. Community licences may carry naming requirements for derivative models and acceptable-use policies that bind your downstream users. Read them once, write the consequences into the card, and the question is answered permanently."}]}],"check":{"question":[{"text":"Which limitations entry is actually useful to somebody deciding whether to rely on the model?"}],"options":[{"text":[{"text":"“This model may occasionally produce inaccurate or inappropriate output and should be used with human oversight.”"}],"correct":false,"tempting":[{"text":"Is true of every model ever released, so it distinguishes nothing and supports no decision."}]},{"text":[{"text":"“The model was evaluated on a held-out set and performs well on the intended task.”"}],"correct":false,"tempting":[{"text":"Restates the evaluation section as a limitation, which tells a reader nothing about where the model fails."}]},{"text":[{"text":"“Training data ends 2026-08; confidence above 0.9 is uncalibrated and must not be thresholded.”"}],"correct":true,"tempting":null},{"text":[{"text":"“Performance may vary depending on input quality, domain and phrasing, as with all language models.”"}],"correct":false,"tempting":[{"text":"Gestures at real sensitivities without naming any of them specifically enough to act on."}]}],"order":[3,2,0,1],"why":null},"prev":{"slug":"versioning-models-and-data","title":"Versioning so a model can be explained later","minutes":8},"next":{"slug":"retiring-a-fine-tune","title":"Retiring a fine-tune","minutes":9}}}