Rebuild the structure first, then the texture

Why big removals fail

Small removals succeed because the surroundings are uniform. A spot on a wall is surrounded by wall.

A large removal (a person in front of a doorway, a car on a street, a pylon across a horizon) fails because the surroundings are structured. Content-aware fill smears, generative fill invents, and cloning leaves a repeated patch, all for the same reason: none of them knows what the building’s geometry was.

The method that works reverses the usual order. Rebuild the structure first, by hand, then borrow texture to cover it.

What content-aware fill is actually doing

It is worth knowing so you can predict it. The underlying algorithm in most implementations is a patch-based synthesis: the best-known is PatchMatch, published in 2009. For every small patch in the hole, it searches the rest of the image for the most similar patch and copies it in, then iterates so neighbouring patches agree with each other.

Everything it puts in was somewhere else in your photograph. It has no model of the world, only of your pixels.

So the failures are predictable:

• Unique content cannot be reconstructed. Remove a person standing in front of a doorway and it fills with wall, because there is no other doorway to copy.
• Straight lines break. A window frame, a skirting board, a horizon continues only if a similarly-angled fragment exists nearby, and the patch search does not understand “continue this line”.
• Perspective is ignored. It copies a patch of paving from the foreground into the middle distance at the wrong scale.
• Repeating textures come back visibly repeated, because the same good patch is the best match many times over.

The controls that help: Photoshop’s Content-Aware Fill workspace lets you paint which areas it may sample from: exclude the foreground when filling a distant area. GIMP’s Resynthesizer plugin, which predates Adobe’s feature and is free, has similar controls and is good.

The manual method, in order

Take a photograph of a street with a parked van you need gone.

1. Reconstruct the large geometry. Identify the lines that must continue through the hole: the kerb, the wall base, the roof line, the far pavement edge. For each, find a clean section elsewhere in the frame, copy it onto a new layer, and move it into position. Use Edit > Free Transform with perspective to match the angle. You are not worrying about texture yet; you are re-establishing where things are.

2. Use the vanishing point or perspective clone for anything receding. Photoshop has Filter > Vanishing Point, where you define a perspective plane and then clone within it, and the tool scales and skews the copied pixels to match. GIMP has the Perspective Clone tool, which does exactly the same job and is one of GIMP’s strong features. Without either, transform a copied patch manually.

3. Fill the remaining flat areas with content-aware fill or Resynthesizer, now that the structure is in place. These tools are good at texture and bad at geometry, so give them only the texture job.

4. Unify the tone. The borrowed pieces came from different parts of the frame with different brightness. A masked Curves layer over the repaired region, matched against its surroundings, removes the patchwork look. This step is skipped more than any other and it is what makes a repair invisible.

5. Add back grain and noise. Cloned and synthesised areas are cleaner than the photograph around them. A noise layer at 1 to 3 per cent, masked to the repaired region and matched to the image’s own grain, is the final tell removed. The compositing block covers matching grain properly.

6. Check at 100 per cent and at fit-to-screen. Repairs that look fine close up often show as a differently-toned block at small size, and repairs that look fine small often have obvious repeats close up.

Generative fill: where it fits

A diffusion model asked to fill a masked region will produce plausible new content rather than borrowed content. For a removal it is often the fastest route, and it comes with the constraint stated across this course: it is inventing.

Two practical notes beyond the honesty question.

It works best on organic, low-stakes content: foliage, sky, gravel, water, crowd. It works worst on anything with regular geometry or text, where it produces convincing-looking nonsense: a shopfront with letters that are not letters, brickwork with courses that do not line up.

Resolution. Most generative fill implementations work on a limited region at a limited resolution and then scale the result in. On a large file the filled area can be softer than its surroundings, which is a giveaway. Filling in several smaller passes rather than one large one helps.

What determines whether you should

The judgement from the earlier retouching lesson, applied to size: does removing this change what the picture asserts?

A bin in the corner of a wedding photograph, no. A brand’s logo on a competitor’s product in an editorial photograph, yes. A person from a news photograph, absolutely: that has ended careers, and it is the clearest line in the trade.

For commercial work the practical safeguard is a note in the delivery: “vehicle removed from left of frame, background reconstructed.” One line, and nobody is surprised later.

The limit

Some removals are not possible, and recognising them early is worth more than technique. The test: is what was behind the object visible anywhere else in the frame, or reconstructible from geometry?

If a van covers a shop’s entire frontage and no other frame shows it, the answer is no. What can be produced is an invention that looks like a shop. Whether that is acceptable depends entirely on what the picture is for, and the person to ask is the client, before you spend the afternoon.