The prompt vocabulary was written by whoever captioned the training clips

Where the training data came from

A video model is trained on tens or hundreds of millions of clips, each paired with a text description. Almost none of those descriptions were written by a person.

The pipeline, in the order it runs:

1. Scrape and segment. Long videos are cut at shot boundaries by a scene-detection tool, so the training unit is a single continuous shot: usually a few seconds. This alone shapes the model: it has essentially never seen a cut.
2. Filter. Clips are scored and discarded. Aesthetic scorers remove ugly ones. Optical-flow scorers remove clips that are too static to teach motion and clips that are too chaotic to learn from. Text detectors often remove clips with heavy on-screen graphics. Watermark detectors remove the obvious ones and miss the rest.
3. Caption. A vision-language model writes a description of each surviving clip. Sometimes two, at different lengths.
4. Train. The model learns the mapping from those captions to those clips.

Everything you experience as the model’s taste was set in steps 2 and 3.

Why mood words fail and shot words work

An auto-captioner describes what it sees. It writes things like “a slow dolly-in on a woman sitting at a desk, warm window light from the left, shallow depth of field”. It does not write “epic” or “masterpiece” or “8k”, because those are not observations.

So those words appear in training text only where a human-written description leaked in, scattered across wildly unrelated footage. Ask for them and you get the average of that scatter.

Meanwhile the captioner’s own vocabulary (shot sizes, camera moves, light direction, lens character) appears tens of millions of times, attached precisely to the thing it names. That is the vocabulary that steers.

The practical rule: write the caption you would want a machine to produce for the clip you want. That is literally the function the model learned.

The stock-footage prior

The filtering stage has consequences that show up in every default the model has.

• It loves a slow push-in. Static clips were filtered out for having too little motion; frantic ones for having too much. What survives the middle of that distribution is a great deal of gentle, drifting, tripod-and-slider stock footage. That is why an under-specified prompt returns a slow dolly.
• It centres the subject. Stock footage composes safely.
• It is landscape. Most of the corpus is 16:9. Vertical performance on many models is visibly weaker, not because 9:16 is harder but because there was less of it, and much of what existed was phone footage of lower quality.
• It grades teal and orange. That look dominates the commercial footage that scores well aesthetically.
• It has a narrow idea of who is in a shot. Stock libraries over-represent some people and settings and under-represent others. Ask for a specific profession or setting and check what you are handed, because the average is somebody else’s average.

None of this is a conspiracy. It is a filter applied at scale, and filters have shapes.

Why the model cannot cut

Worth stating on its own. Scene detection meant every training clip was one unbroken shot. The model has no representation of a cut, which is why asking for “a montage” or “then it cuts to” produces either a morph or a camera move rather than an edit.

Cuts are your job, in the editor. That is not a limitation to work around; it is the correct division of labour, and it is why the edit module exists.

Recaptioning, and why prompts stopped needing to be weird

Several labs now recaption their entire corpus with a better VLM before training, in a consistent style and at a consistent length. The effect on users is direct: prompts that read like ordinary descriptive English work better than keyword soup, because the training text was ordinary descriptive English.

If a model’s documentation shows long, prose-style example prompts, that is a recaptioned model and you should write prose. If the examples are comma-separated keyword lists, write keyword lists. The examples are not marketing: they are a sample of the training distribution, and matching it is worth more than any prompt trick.

The free path to finding a model’s vocabulary

You do not need to guess. Take one still and generate it six times with six different camera-move phrasings, one word changed each time. Keep the ones that produced a distinct, correct move. That list of six or ten phrases is your working vocabulary for that model, and it will be more reliable than any prompt guide written for a different one.

This costs a free daily allowance and about twenty minutes, and it is the highest-value use of a free tier there is.

Today: read the example prompts in your model’s own documentation, and count how many are mood adjectives versus observations. Write your next prompt in whichever style dominates.