When a LoRA is worth the afternoon

What a LoRA is, in one paragraph

A LoRA (low-rank adaptation) is a small set of extra weights trained on top of a frozen base model. Instead of retraining billions of parameters, you train a few million that nudge the model’s behaviour toward one concept. The file is typically tens to a few hundred megabytes and loads alongside the base model at generation time.

For identity, this is the difference in kind. A reference points toward a region. A LoRA teaches the model a concept, which then behaves like every other concept it knows: it can be posed, lit, dressed, put at a different angle, and it stays itself.

The dataset, which is where the quality comes from

Fifteen to thirty images is the usual range. More is not automatically better; varied and clean beats numerous.

What the set needs:

• Varied angles. Frontal, three-quarter both sides, profile, slightly above, slightly below.
• Varied expressions. Neutral, speaking, a small smile. A set of thirty neutral frontals produces a model that can only do one face.
• Varied framing. Close-ups and at least a few full-body shots, or the model will not know what the body looks like.
• Varied backgrounds and light. This is the opposite of the character sheet rule and it is deliberate. If every training image has the same background, the model learns the background as part of the identity. Variety is what teaches it which parts are the person.
• Consistent identity. Every image must be the same person. One wrong face poisons the set.

Captions matter. Caption what varies and not what is constant. If every image is your character, do not describe her face in every caption: describe the angle, the setting, the clothing. What you leave undescribed is what the trigger word absorbs.

The training run

On video models, the practical route today is usually to train the identity on the image side and use those images as first frames, or to train a LoRA for an open-weights video model where adapter support exists.

Rough shape for an image LoRA, which is the accessible case:

• 20 images, 1500 to 3000 training steps, a learning rate around 1e-4, rank 16 to 32.
• Twenty to sixty minutes on a mid-range GPU.
• Free on Colab or Kaggle for smaller models; several Hugging Face Spaces offer one-click training.
• Tools: kohya_ss, diffusers training scripts, or a ComfyUI trainer node.

Overtraining is the common failure and it has a recognisable signature: the character appears in the right pose and the right clothes in every image, because the model memorised the training photographs rather than the person. If your outputs all look like your dataset, train fewer steps or gather more varied images.

When it is worth it

Honest arithmetic. A LoRA costs you: gathering a dataset, a training run, and the local setup. Call it half a day the first time and an hour once you have done it before.

It repays when:

• The character appears in more than roughly ten shots.
• The face is visible and close in several of them.
• The project will have revisions, so shots must be rebuildable months later.
• You will reuse the character across multiple projects: a brand mascot, a recurring presenter, a series.

It does not repay for a three-shot social clip, a background character, or anything where a reference feature already gets you close enough.

The part that is not technical

If the character is based on a real person, you need their written permission, specific to this use, before you build a dataset of their face. A trained model of somebody’s likeness is a more consequential object than a single generated image: it is a reusable capability to produce new pictures of them indefinitely.

Treat it accordingly. Say what it will be used for, for how long, and what happens to the file afterwards. “We will delete the model at the end of the campaign” is a commitment you can make and should keep.

This is dealt with properly in the module on law and disclosure. It is raised here because the technical lesson is where the temptation arrives.

The free path

Genuinely complete. Open base models, free training scripts, free GPU sessions on Colab and Kaggle, and Hugging Face Spaces that will train a small LoRA from an upload. The binding constraint is a session that does not disconnect, which is the usual cost of free compute.

More general ground in fine-tuning and running-models-yourself.

Today: count how many shots in your project show one character’s face at medium or closer. If the number is above ten, you have your answer about whether to train.