{"course":{"slug":"ai-video","title":"Video with AI","url":"https://addaly.com/learn/ai-video","level":"intermediate","track":"create"},"lesson":{"slug":"reference-and-element-conditioning","title":"What a reference image is actually doing","url":"https://addaly.com/learn/ai-video/reference-and-element-conditioning","position":32,"of":66,"minutes":9,"answer":"Reference features condition on an embedding of the supplied images rather than storing the person, so they reliably reproduce a type (build, colouring, wardrobe.","keyIdea":"Reference features condition on an embedding of the supplied images rather than storing the person, so they reliably reproduce a type (build, colouring, wardrobe, general impression) while the exact facial geometry keeps re-sampling within that type."},"body":{"format":"markdown","text":"## The feature, under several names\n\nKling calls them elements. Runway calls them references. Higgsfield has character tools. Sora has consent-gated cameos. Image models have their own versions, which matter here because your first frames come from them.\n\nThe common mechanism: you supply one or more images of a subject, an encoder turns them into a conditioning vector, and that vector is injected alongside the text and the first frame. The model is pulled toward the region of its output space that matches the reference.\n\nThat phrasing (*pulled toward a region*) is the whole lesson. It is not retrieval and it is not copying. There is no stored image of your person being pasted in.\n\n## What this gets you, reliably\n\n- Build, height, apparent age.\n- Skin tone, hair colour, hair length and general shape.\n- Wardrobe colour and rough garment type.\n- The overall impression somebody would describe in a sentence.\n\nThis is valuable. A reference gets you a consistent *type* of person across shots, which is far better than a description, and for background characters, crowds and anyone who is not the lead it is entirely sufficient.\n\n## What it does not get you\n\nThe exact face. Over several shots, the facial geometry re-samples within the region the reference defines. Eye spacing, nose width, jaw shape and the relationship between features shift by a few per cent each time.\n\nA few per cent is under the threshold for a chair and well over it for a face. This is the gap between \"a woman who looks like her\" and \"her\", and it is where character-consistency demos stop being impressive if you watch them carefully.\n\nThe honest formulation: references give you a consistent character *design*. They do not give you a consistent *actor*.\n\n> Figure: A reference gives you a design, not an actor. Reliably reproduced: Build, height and apparent age; Skin tone, hair colour, hair length and general shape; Wardrobe colour and rough garment type; The impression somebody would give in one sentence. Re-sampled every generation: Eye spacing, nose width and jaw shape, by a few per cent each time; The relationship between features, which is what identity is read from; Anything fine: glasses frames, a small mole, a thin chain; The reference's own lighting, which comes in along with the face. A few per cent of change is under the threshold for a chair and well over it for a face, which is exactly where the character-consistency demos stop being impressive if you watch them carefully. Use the reference to build a good first frame, then generate from the frame.\n\n## Getting the most out of them\n\nFive things that measurably help:\n\n1. **Multiple reference images, varied angles, consistent light.** Your character sheet is exactly this. One frontal image produces a model that only knows the front.\n2. **Plain backgrounds in the references.** A busy background leaks into the conditioning. The model has no clean separation between \"the person\" and \"the pixels around the person\".\n3. **Reference plus first frame, not reference alone.** The first frame is still the stronger signal. Use the reference to build a good first frame, then generate from that frame, and the reference has done its most valuable work before the video model starts.\n4. **Do not mix references of different people.** Two subjects in one conditioning produces a blend, and the blend is stable and wrong.\n5. **Crop the references consistently.** Head-and-shoulders references produce head-and-shoulders influence. If you need the wardrobe held, include a full-body reference.\n\n## Where the leakage happens\n\nA specific, common frustration worth naming. Supply a reference photographed outdoors at midday, then ask for an indoor evening shot. The output comes back oddly bright and cool, because the reference carried its lighting in with it.\n\nThe conditioning encodes the whole image, not a disentangled notion of identity. Same with background colour, same with the camera's apparent lens.\n\nSo make your references under the light you intend to use, which your character sheet already specifies. If you need the character under two different lights, make two sheets.\n\n## Consent-gated references\n\nSome products require a person to record a verification video before their likeness can be used: a short clip, live, proving they are the person and that they agree. Sora's cameo mechanism is the best-known example.\n\nThis is a good design and it is worth understanding as a norm rather than as a feature. It moves consent from a checkbox you tick about somebody else to an act that person performs themselves. Whether a given product implements it well is a separate question; the shape is right.\n\nIt does not replace your own paperwork on a commercial job, which is dealt with later in the course.\n\n## The free path\n\nOpen-weights image pipelines expose reference conditioning (IP-Adapter and its descendants) in ComfyUI, free, and several Hugging Face Spaces demonstrate it in a browser. On the video side, reference features are currently more common on hosted tools, so the free route is to do the reference work at the still stage, where the free options are strong, and let the video model animate the frame you built.\n\nThat sequencing is not a workaround. It is the better workflow for cost reasons anyway.\n\n---\n\nToday: generate the same shot twice with the same reference images and compare the two faces at full size, side by side. The gap you see is the gap you will be managing all project."},"check":{"question":"A reference image photographed outdoors at midday is used to generate an indoor evening shot, and the result comes back too bright and too cool. Why?","options":[{"text":"The conditioning encodes the whole reference image, including its lighting, rather than a disentangled representation of the person.","correct":true},{"text":"Reference conditioning is applied after the text conditioning in the pipeline, so it overrides any lighting described in the prompt.","correct":false,"tempting":"Invents an ordering rule; the two are combined, and the prompt's lighting is weakened rather than overridden outright."},{"text":"Daylight-balanced images have a wider colour gamut, so the encoder assigns them higher weight than the darker regions the prompt requested.","correct":false,"tempting":"Invokes gamut and weighting in a way the encoder does not work, and would predict the same problem in reverse, which does not occur."},{"text":"The model interprets a high-key reference as a request for high-key output, because aesthetic filtering during training favoured brighter clips and biased the association.","correct":false,"tempting":"Names a plausible dataset bias, but the effect persists on models with different filtering and tracks the specific reference rather than a general preference."}]},"prev":{"slug":"building-a-character-sheet","title":"The document you build before you generate anything","url":"https://addaly.com/learn/ai-video/building-a-character-sheet"},"next":{"slug":"training-a-character-model","title":"When a LoRA is worth the afternoon","url":"https://addaly.com/learn/ai-video/training-a-character-model"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What a reference image is actually doing', in Video with AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-video/reference-and-element-conditioning (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What a reference image is actually doing","courseTitle":"Video with AI","module":{"index":4,"slug":"continuity-and-character","title":"Making separate shots belong to each other"},"answer":null,"keyIdea":"Reference features condition on an embedding of the supplied images rather than storing the person, so they reliably reproduce a type (build, colouring, wardrobe, general impression) while the exact facial geometry keeps re-sampling within that type.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"left":{"title":"Reliably reproduced","items":["Build, height and apparent age","Skin tone, hair colour, hair length and general shape","Wardrobe colour and rough garment type","The impression somebody would give in one sentence"]},"type":"compare","right":{"title":"Re-sampled every generation","items":["Eye spacing, nose width and jaw shape, by a few per cent each time","The relationship between features, which is what identity is read from","Anything fine: glasses frames, a small mole, a thin chain","The reference’s own lighting, which comes in along with the face"]},"title":"A reference gives you a design, not an actor","caption":"A few per cent of change is under the threshold for a chair and well over it for a face, which is exactly where the character-consistency demos stop being impressive if you watch them carefully. Use the reference to build a good first frame, then generate from the frame."},"alt":"Figure: A reference gives you a design, not an actor. Reliably reproduced: Build, height and apparent age; Skin tone, hair colour, hair length and general shape; Wardrobe colour and rough garment type; The impression somebody would give in one sentence. Re-sampled every generation: Eye spacing, nose width and jaw shape, by a few per cent each time; The relationship between features, which is what identity is read from; Anything fine: glasses frames, a small mole, a thin chain; The reference’s own lighting, which comes in along with the face. A few per cent of change is under the threshold for a chair and well over it for a face, which is exactly where the character-consistency demos stop being impressive if you watch them carefully. Use the reference to build a good first frame, then generate from the frame."}},"blocks":[{"t":"h2","id":"the-feature-under-several-names","runs":[{"text":"The feature, under several names"}]},{"t":"p","runs":[{"text":"Kling calls them elements. Runway calls them references. Higgsfield has character tools. Sora has consent-gated cameos. Image models have their own versions, which matter here because your first frames come from them."}]},{"t":"p","runs":[{"text":"The common mechanism: you supply one or more images of a subject, an encoder turns them into a conditioning vector, and that vector is injected alongside the text and the first frame. The model is pulled toward the region of its output space that matches the reference."}]},{"t":"p","runs":[{"text":"That phrasing ("},{"text":"pulled toward a region","i":true},{"text":") is the whole lesson. It is not retrieval and it is not copying. There is no stored image of your person being pasted in."}]},{"t":"h2","id":"what-this-gets-you-reliably","runs":[{"text":"What this gets you, reliably"}]},{"t":"ul","items":[[{"text":"Build, height, apparent age."}],[{"text":"Skin tone, hair colour, hair length and general shape."}],[{"text":"Wardrobe colour and rough garment type."}],[{"text":"The overall impression somebody would describe in a sentence."}]]},{"t":"p","runs":[{"text":"This is valuable. A reference gets you a consistent "},{"text":"type","i":true},{"text":" of person across shots, which is far better than a description, and for background characters, crowds and anyone who is not the lead it is entirely sufficient."}]},{"t":"h2","id":"what-it-does-not-get-you","runs":[{"text":"What it does not get you"}]},{"t":"p","runs":[{"text":"The exact face. Over several shots, the facial geometry re-samples within the region the reference defines. Eye spacing, nose width, jaw shape and the relationship between features shift by a few per cent each time."}]},{"t":"p","runs":[{"text":"A few per cent is under the threshold for a chair and well over it for a face. This is the gap between “a woman who looks like her” and “her”, and it is where character-consistency demos stop being impressive if you watch them carefully."}]},{"t":"p","runs":[{"text":"The honest formulation: references give you a consistent character "},{"text":"design","i":true},{"text":". They do not give you a consistent "},{"text":"actor","i":true},{"text":"."}]},{"t":"h2","id":"getting-the-most-out-of-them","runs":[{"text":"Getting the most out of them"}]},{"t":"p","runs":[{"text":"Five things that measurably help:"}]},{"t":"ol","start":1,"items":[[{"text":"Multiple reference images, varied angles, consistent light.","b":true},{"text":" Your character sheet is exactly this. One frontal image produces a model that only knows the front."}],[{"text":"Plain backgrounds in the references.","b":true},{"text":" A busy background leaks into the conditioning. The model has no clean separation between “the person” and “the pixels around the person”."}],[{"text":"Reference plus first frame, not reference alone.","b":true},{"text":" The first frame is still the stronger signal. Use the reference to build a good first frame, then generate from that frame, and the reference has done its most valuable work before the video model starts."}],[{"text":"Do not mix references of different people.","b":true},{"text":" Two subjects in one conditioning produces a blend, and the blend is stable and wrong."}],[{"text":"Crop the references consistently.","b":true},{"text":" Head-and-shoulders references produce head-and-shoulders influence. If you need the wardrobe held, include a full-body reference."}]]},{"t":"h2","id":"where-the-leakage-happens","runs":[{"text":"Where the leakage happens"}]},{"t":"p","runs":[{"text":"A specific, common frustration worth naming. Supply a reference photographed outdoors at midday, then ask for an indoor evening shot. The output comes back oddly bright and cool, because the reference carried its lighting in with it."}]},{"t":"p","runs":[{"text":"The conditioning encodes the whole image, not a disentangled notion of identity. Same with background colour, same with the camera’s apparent lens."}]},{"t":"p","runs":[{"text":"So make your references under the light you intend to use, which your character sheet already specifies. If you need the character under two different lights, make two sheets."}]},{"t":"h2","id":"consent-gated-references","runs":[{"text":"Consent-gated references"}]},{"t":"p","runs":[{"text":"Some products require a person to record a verification video before their likeness can be used: a short clip, live, proving they are the person and that they agree. Sora’s cameo mechanism is the best-known example."}]},{"t":"p","runs":[{"text":"This is a good design and it is worth understanding as a norm rather than as a feature. It moves consent from a checkbox you tick about somebody else to an act that person performs themselves. Whether a given product implements it well is a separate question; the shape is right."}]},{"t":"p","runs":[{"text":"It does not replace your own paperwork on a commercial job, which is dealt with later in the course."}]},{"t":"h2","id":"the-free-path","runs":[{"text":"The free path"}],"role":"free"},{"t":"p","runs":[{"text":"Open-weights image pipelines expose reference conditioning (IP-Adapter and its descendants) in ComfyUI, free, and several Hugging Face Spaces demonstrate it in a browser. On the video side, reference features are currently more common on hosted tools, so the free route is to do the reference work at the still stage, where the free options are strong, and let the video model animate the frame you built."}]},{"t":"p","runs":[{"text":"That sequencing is not a workaround. It is the better workflow for cost reasons anyway."}]},{"t":"rule"},{"t":"p","runs":[{"text":"Today: generate the same shot twice with the same reference images and compare the two faces at full size, side by side. The gap you see is the gap you will be managing all project."}]}],"check":{"question":[{"text":"A reference image photographed outdoors at midday is used to generate an indoor evening shot, and the result comes back too bright and too cool. Why?"}],"options":[{"text":[{"text":"The conditioning encodes the whole reference image, including its lighting, rather than a disentangled representation of the person."}],"correct":true,"tempting":null},{"text":[{"text":"Reference conditioning is applied after the text conditioning in the pipeline, so it overrides any lighting described in the prompt."}],"correct":false,"tempting":[{"text":"Invents an ordering rule; the two are combined, and the prompt’s lighting is weakened rather than overridden outright."}]},{"text":[{"text":"Daylight-balanced images have a wider colour gamut, so the encoder assigns them higher weight than the darker regions the prompt requested."}],"correct":false,"tempting":[{"text":"Invokes gamut and weighting in a way the encoder does not work, and would predict the same problem in reverse, which does not occur."}]},{"text":[{"text":"The model interprets a high-key reference as a request for high-key output, because aesthetic filtering during training favoured brighter clips and biased the association."}],"correct":false,"tempting":[{"text":"Names a plausible dataset bias, but the effect persists on models with different filtering and tracks the specific reference rather than a general preference."}]}],"order":[3,2,0,1],"why":null},"prev":{"slug":"building-a-character-sheet","title":"The document you build before you generate anything","minutes":9},"next":{"slug":"training-a-character-model","title":"When a LoRA is worth the afternoon","minutes":10}}}