Model cards and disclosure

The document that answers the questions

A model card is to a model what a data card is to a dataset: the page a stranger reads to decide whether they can rely on it. It is also, in practice, the page that answers procurement questionnaires, security reviews and the first hour of an incident.

It takes an hour to write and it is written once per major version.

# triage-3.2.0

## What it does
Routes a customer message to one of six queues and drafts a one-paragraph
reply. English only.

## Base and licence
Llama-3.1-8B-Instruct @ 0e9e39f, under the Llama 3.1 Community Licence.
Naming and acceptable-use terms follow this derivative.
Adapter: LoRA r=16, all-linear. Merged and quantised to Q4_K_M.

## Training data
ticket-triage-sft@v3 — 31,084 examples: 24,003 internal support logs
(ToS v4 basis, PII-redacted), 2,140 hand-written, 4,941 OpenAssistant
(Apache-2.0) as replay. Data card: docs/data/ticket-triage-v3.md

## Evaluation
Held-out n=412, drawn after the training cutoff, group-split by customer.
  routing accuracy   0.918  [0.888 – 0.941]
  valid JSON         1.000  (constrained decoding)
  canary (general)   0.812  (base 0.840)
  refusal set        0.94   (base 0.96)
Seed noise floor: sd 1.3 points over 3 runs.
Judged by: exact match on queue; replies by human pairwise, n=200.

## Intended use
First-line triage with a human reviewing anything the model routes to
'refunds' or 'escalation'.

## Out of scope
Not for medical, legal or financial advice. Not for non-English input.
Not for deciding refunds. Not an identity or eligibility check.

## Known limitations
- No data after 2026-08; the September pricing change is not represented.
- 4% of live traffic is non-English and is routed to 'other' by design.
- Canary is 2.8 points below base: general capability is slightly reduced.
- Confidence is uncalibrated above 0.9; do not threshold there.

## Contact
r.kapoor

The sections that earn their space

Out of scope. The most useful section and the hardest to write, because it requires saying no to uses somebody has already imagined. It is also the section that protects you, because a stated boundary is a decision rather than an oversight.

Known limitations, with mechanisms. Not “may occasionally be inaccurate” (which is true of everything and informs nobody) but the specific, mechanical limits: the data cutoff, the languages, the canary gap, the calibration range.

Evaluation with intervals and provenance. A number without an interval and without a statement of what judged it is not a result.

Disclosure to users

Separate from the card, and simpler: if a person is interacting with a model, say so, plainly, where they are looking. Do not let a name, an avatar or a writing style imply a person.

This is the right thing regardless of what any regulation requires, and it is also the minimum several regulations are converging on.

The regulatory shape, without advice

The landscape differs sharply by jurisdiction and is moving, so what follows is the shape of it rather than a statement of what you must do.

• The European Union’s AI Act takes a risk-tiered approach with transparency obligations (disclosing that a person is interacting with an AI system, marking synthetic content) plus documentation duties on providers of general-purpose models, phasing in over several years. How its obligations fall on somebody who fine-tunes a third party’s model is a question with guidance still being worked out.
• Sectoral regulators in many countries (financial, medical, employment) have applied existing rules to automated decisions without new AI-specific legislation, and those rules can be stricter than any AI act.
• Several jurisdictions have synthetic-media and labelling rules at various stages, and they do not agree with each other.
• Many countries have nothing specific at all, where general consumer-protection, data-protection and anti-discrimination law applies as it always did.

So: this is unsettled, it depends on where you operate and what the system decides, and I am not in a position to tell you what applies to you. What travels everywhere is the documentation itself. A card with intended use, out-of-scope uses, training-data provenance, evaluation results and stated limitations is what any of these regimes asks for in some form, and it is what you would want to have written regardless.

Inherited terms

One practical item people miss: your base model’s licence terms generally follow into your derivative. Community licences may carry naming requirements for derivative models and acceptable-use policies that bind your downstream users. Read them once, write the consequences into the card, and the question is answered permanently.