What a reference image is actually doing

The feature, under several names

Kling calls them elements. Runway calls them references. Higgsfield has character tools. Sora has consent-gated cameos. Image models have their own versions, which matter here because your first frames come from them.

The common mechanism: you supply one or more images of a subject, an encoder turns them into a conditioning vector, and that vector is injected alongside the text and the first frame. The model is pulled toward the region of its output space that matches the reference.

That phrasing (pulled toward a region) is the whole lesson. It is not retrieval and it is not copying. There is no stored image of your person being pasted in.

What this gets you, reliably

• Build, height, apparent age.
• Skin tone, hair colour, hair length and general shape.
• Wardrobe colour and rough garment type.
• The overall impression somebody would describe in a sentence.

This is valuable. A reference gets you a consistent type of person across shots, which is far better than a description, and for background characters, crowds and anyone who is not the lead it is entirely sufficient.

What it does not get you

The exact face. Over several shots, the facial geometry re-samples within the region the reference defines. Eye spacing, nose width, jaw shape and the relationship between features shift by a few per cent each time.

A few per cent is under the threshold for a chair and well over it for a face. This is the gap between “a woman who looks like her” and “her”, and it is where character-consistency demos stop being impressive if you watch them carefully.

The honest formulation: references give you a consistent character design. They do not give you a consistent actor.

Getting the most out of them

Five things that measurably help:

1. Multiple reference images, varied angles, consistent light. Your character sheet is exactly this. One frontal image produces a model that only knows the front.
2. Plain backgrounds in the references. A busy background leaks into the conditioning. The model has no clean separation between “the person” and “the pixels around the person”.
3. Reference plus first frame, not reference alone. The first frame is still the stronger signal. Use the reference to build a good first frame, then generate from that frame, and the reference has done its most valuable work before the video model starts.
4. Do not mix references of different people. Two subjects in one conditioning produces a blend, and the blend is stable and wrong.
5. Crop the references consistently. Head-and-shoulders references produce head-and-shoulders influence. If you need the wardrobe held, include a full-body reference.

Where the leakage happens

A specific, common frustration worth naming. Supply a reference photographed outdoors at midday, then ask for an indoor evening shot. The output comes back oddly bright and cool, because the reference carried its lighting in with it.

The conditioning encodes the whole image, not a disentangled notion of identity. Same with background colour, same with the camera’s apparent lens.

So make your references under the light you intend to use, which your character sheet already specifies. If you need the character under two different lights, make two sheets.

Consent-gated references

Some products require a person to record a verification video before their likeness can be used: a short clip, live, proving they are the person and that they agree. Sora’s cameo mechanism is the best-known example.

This is a good design and it is worth understanding as a norm rather than as a feature. It moves consent from a checkbox you tick about somebody else to an act that person performs themselves. Whether a given product implements it well is a separate question; the shape is right.

It does not replace your own paperwork on a commercial job, which is dealt with later in the course.

The free path

Open-weights image pipelines expose reference conditioning (IP-Adapter and its descendants) in ComfyUI, free, and several Hugging Face Spaces demonstrate it in a browser. On the video side, reference features are currently more common on hosted tools, so the free route is to do the reference work at the still stage, where the free options are strong, and let the video model animate the frame you built.

That sequencing is not a workaround. It is the better workflow for cost reasons anyway.

Today: generate the same shot twice with the same reference images and compare the two faces at full size, side by side. The gap you see is the gap you will be managing all project.