Writing a card somebody can audit

Two audiences, one document

A model card is read by somebody deciding whether to use your model, and, increasingly, by somebody asking how a system in production was built. The second reader is new, and they ask harder questions: what was it trained on, how was it tested, who is accountable, what does it do to people it was not designed for.

Writing for the second reader improves the document for the first, because the questions are the same ones a careful user has and does not ask.

The structure

Front matter that is correct. license, base_model, language, datasets, pipeline_tag, library_name. These are indexed, they make your work findable, and a wrong pipeline_tag means nobody filtering for your task ever sees it.

One paragraph: what it is and what it is for. No marketing. “A LoRA adapter for Qwen2.5-7B-Instruct that classifies Hindi and English customer support messages into nine intents.”

A runnable snippet. With the revision pinned. Test it in a fresh environment before publishing: the commonest defect in cards is a snippet that never ran.

Intended use, and out-of-scope use. The second is the one people skip and the one that protects everybody. “Not for medical, legal or financial advice. Not evaluated on languages other than Hindi and English. Not suitable for deciding anything about a person without human review.”

Training data. Source, size, date range, languages, how it was collected, and whether it contained personal data. If you cannot publish the dataset, describe it. A card that says nothing about data is a card that cannot be assessed.

Training procedure. Base model and revision, method, hyperparameters, hardware, duration. Six lines, and they are what makes your work reproducible rather than anecdotal.

Evaluation. What you measured, on what, against what baseline. A number without a baseline means nothing; a number without a described set means less.

Limitations and bias. This is the section that separates a card from a README, and it has to be specific. Not “may produce biased output”, which is true of everything and informs nobody, but “accuracy on Hindi messages is 8 points below English; the training data came from one company’s support queue in 2024 and skews toward delivery complaints; the model has not been tested on Hinglish code-switching, which is common in this domain.”

Environmental cost, if you can compute it. Hardware, hours, and where it ran. One line.

Contact. A discussion tab is enough, but say so.

Per-group results, where they apply

If your model touches people, one overall number hides what matters. Report the metric separately by whatever groups your data supports: language, region, input length, customer type. A model at 0.89 overall may be at 0.93 on English and 0.71 on Hindi, and the second pair of numbers is the whole story for a service whose users are mostly Hindi speakers.

You can only report groups your data records. That is itself a finding worth stating: “we could not measure performance by region because region is not recorded” is honest and useful.

What regulation is starting to ask for

Rules are arriving at different speeds in different places. The European Union’s AI Act introduces obligations that vary by risk category and by whether you are a provider or a deployer, with documentation and transparency duties among them. Other jurisdictions are drafting, consulting or doing nothing. Sectoral rules (medical devices, financial advice, employment decisions) often bite before any AI-specific law does.

The honest summary is that the obligations differ by where you are, what the system does, and who uses it, and that this is moving. Nothing here is legal advice. What generalises is the practice: a documented dataset, a stated intended use, a measured evaluation with per-group results, a named limitation and a way to contact somebody. A team that has those can answer whatever the specific requirement turns out to be. A team that has none of them is starting from nothing whenever somebody asks.

The test

Give the card to somebody who has not seen the project. Ask them: what does it do, what will it do badly, can they run it in ten minutes, and would they be comfortable if it were used on them. Those four questions catch nearly everything.

Do this now

Open your best existing card and write the out-of-scope section, specifically, from your own testing. It is the section you can write today with information you already have, and it is the one most likely to stop somebody misusing your work.