Three equations, seven unknowns: why hair is hard

The compositing equation

Every pixel in a photograph where subject meets background obeys one relationship:

I = α·F + (1 − α)·B

I is what the camera recorded. F is the foreground’s true colour at that pixel, B is the background’s true colour, and α is what fraction of the pixel the foreground covers.

That is three equations, one per colour channel, and seven unknowns: three for F, three for B, and α. The system is underdetermined. There is no correct answer derivable from the pixel alone; there are infinitely many combinations of foreground, background and coverage that produce the same recorded value.

This is not a software limitation. It is why extracting hair is hard, why every tool needs extra information from you, and why the answer is always an estimate.

How the extra information is supplied

A trimap. You mark three regions: definitely foreground, definitely background, and unknown. The algorithm now has known F values nearby and known B values nearby, and only has to solve for α in the unknown band. This is exactly what the Refine Edge brush in Select and Mask is doing when you scrub over the hair: you are drawing the unknown region of a trimap.

A known background. Shoot against a uniform colour and B is known everywhere, which collapses the problem. That is chroma key, and it is why studios use it.

A learned prior. Deep matting models estimate α from the image using patterns learned from thousands of hand-made mattes. Better than the classical methods on hair and confidently wrong on anything unusual.

Colour spill, and why contracting does not fix it

Now the second half of the problem. The equation says a partial pixel’s recorded colour I is a mixture that includes the background. Extract the alpha correctly and place the subject on white, and those partial pixels still carry the old background’s colour in their F.

Against a green wall, hair edges are green. Against a red curtain, they are pink. Against a grey studio wall, they are grey, which is the familiar dull halo.

Three things people try, and why each fails:

• Contracting the selection by a pixel. This does not remove contamination; it removes the hair. Fine strands are only two or three pixels wide, so contracting deletes them entirely, and you get the helmet.
• Feathering. This blends the contaminated pixels further into the surroundings, spreading the problem over a wider band.
• Raising the mask contrast. This turns partial pixels into fully-on or fully-off ones, which removes the softness that made the edge believable.

The only fix that addresses the cause is colour decontamination: recomputing the F of each partial pixel from the nearby known foreground colour, while keeping its α. In Photoshop this is the Decontaminate Colours checkbox in Select and Mask, and it outputs to a new layer because it rewrites pixel colours. In video compositing it is called spill suppression and every keyer has one.

The free route: GIMP does not have an automatic decontaminator, and the manual equivalent works. Create a new layer clipped to the cut-out, set it to Colour blend mode, sample the subject’s correct colour near the edge, and paint along the contaminated fringe at 30 to 50 per cent opacity. You are replacing the hue while keeping the luminosity and the alpha. It takes two minutes on a portrait and it works.

A second manual trick for green spill specifically: a Hue/Saturation layer targeting Greens, saturation down, masked to the edge band. Crude, fast, and adequate for a marketplace listing.

Shooting to make this easy

The cheapest fix happens before the shutter.

• Shoot against a background close in tone and neutral in colour to what the subject will end up on. Grey or white for a product going on white. Contamination from a neutral background is far less visible than from a coloured one.
• Separate the subject from the background by two metres if you can, and light them separately. This reduces the coloured light bouncing off the background onto the subject, which is spill that no decontamination can fully remove because it is real light that landed on the hair.
• Green screen is for video, not stills. Video needs it because frame-by-frame manual work is impossible. For a single still, a plain grey wall plus ten minutes of masking gives a better result with no spill problem.

Checking the result

The test that catches everything: put the cut-out on three backgrounds in turn: white, mid-grey, and a saturated colour that contrasts with the original background. The saturated one is where contamination screams. If the hair reads as clean on all three, it is clean.

Then look at the edge at 100 per cent against the darkest of them. A pale rim you did not see on white will be obvious on black.

What remains unsolvable

Some pixels have no answer. If the subject is motion-blurred against a background, the mixing happened over time as well as space, and the equation’s F is itself a blur of several different foreground colours. If the background behind the hair is itself detailed and high contrast (a bookshelf, foliage) then the contaminating colour varies pixel to pixel and decontamination has no consistent value to replace it with.

In those cases the honest answer is that the shot cannot be cut out cleanly, and the choices are to keep some of the original background as a soft vignette, to reshoot, or to tell the client. Spending three hours on a shot that was not extractable is a mistake most people make once.