Where the data legitimately comes from

Four sources, in descending order of usefulness

1. Your own traffic. Real inputs from real people doing the real job. Nothing else has the right distribution: the typos, the half-sentences, the questions that do not fit your categories. Sample it deliberately: stratify by intent, language and length rather than taking the last thousand rows, which cover the last twenty minutes of one time zone.

The obligations attach here, not later. Check what your privacy notice promised, redact identifiers before anything lands in a training file, and record the basis on which you are using it. A training set is a copy of your users’ words with the access controls taken off.

2. Adjacent real text. Support tickets, forum threads, internal documentation, past reports: text produced by people doing the task, even if never typed at a model. It has the vocabulary and the register, and it needs conversion into instruction shape, which is work.

3. Team-written examples. Slower, higher quality, and the only source that can cover a case which has never occurred. Twenty hand-written examples at the start of a project are worth more than two thousand scraped ones, because writing them forces you to decide what a good answer actually is, and you will change your mind twice while doing it.

4. Public instruction datasets. Useful as a replay mixture rather than as your task data. Which brings the licence question.

The public sets worth knowing, and their terms

Licences change; check the dataset card for the current terms before you rely on any of this.

• databricks-dolly-15k: 15,000 prompt-and-response pairs written by Databricks employees, released CC BY-SA 3.0. Human-written, so no question about another provider’s output terms. The share-alike condition is the thing to think about.
• OpenAssistant (oasst1, oasst2): human-generated, multilingual, crowd-sourced conversation trees, Apache-2.0. The most permissive substantial human-written set available.
• No Robots: 10,000 human-written instruction-and-demonstration pairs, deliberately produced without model assistance. Released under a non-commercial licence, which is exactly the sort of detail that matters and is routinely overlooked.
• UltraChat and similar large synthetic sets: permissively licensed as datasets, but generated by calling a hosted model, so the generating provider’s terms are a separate question from the dataset’s licence.

That last distinction is the one that catches teams. A dataset’s own licence and the terms under which its contents were produced are two different permissions, and a permissive dataset licence cannot grant something the producer did not have.

The provenance column

One extra field per example, written at collection time, costs nothing and answers every question anybody will later ask:

{"messages": [...],
 "source": "support_logs",
 "licence": "internal",
 "basis": "tos_v4_analytics",
 "collected": "2026-08-14",
 "reviewed_by": "rk"}

Six months later, somebody asks whether the model was trained on customer data. With this column it is a query. Without it, it is an investigation that ends in a guess.

How much of each

A shape that works for a task fine-tune:

• 60 to 80% task data from your own traffic or adjacent text
• 10 to 20% hand-written examples covering the cases traffic does not contain: the edge cases, the refusals, the “I do not have that information” responses
• 10 to 20% general instruction data as replay, to stop the model forgetting how to be a model

The middle slice is the one people skip, and it is the one that determines how the model behaves when something unusual arrives. If every training example is a question the model can answer, you have taught it that every question is answerable. That is exactly the behaviour people call hallucination, and you installed it.

What not to do

Do not scrape a competitor’s product outputs. Do not use a dataset because it is popular without reading its card. Do not take the last hundred thousand log rows because they were easy to get.

And do not let dataset size become the goal. The LIMA result (a strong instruction-following model fine-tuned on a thousand carefully curated examples) is the standing evidence that curation beats volume at this scale. A thousand examples you have read is a better dataset than fifty thousand you have not.