Stop fighting the selection tools and paint it

The mode that makes a selection into an image

Press Q and the marching ants disappear, replaced by a red overlay. You are now in Quick Mask mode, and the selection is being displayed as a temporary greyscale image: selected areas clear, unselected areas red, partially selected areas partly red.

While you are in that mode, every painting tool works on the selection. Paint with black and the red spreads, removing area from the selection. Paint with white and the red is wiped away, adding area. Grey adds partial selection. Use a gradient, a blur, a texture brush: anything that makes marks makes selection.

Press Q again and you are back to a normal selection, ready to become a mask.

GIMP has the identical feature under Select > Toggle Quick Mask or Shift+Q, with the same red overlay. Krita has Selection Masks, which are similar and persist. Photopea has Quick Mask on the same Q shortcut.

Why this changes how the work feels

The selection tools are all attempts to guess a boundary. When the guess is good, they are fast. When the guess is bad (a low-contrast edge, a busy background, an object that shares colour with what is behind it) you can spend twenty minutes fighting tolerance sliders and add-and-subtract clicks.

Quick Mask removes the guessing. The rough selection from Quick Selection gets you eighty per cent of the way in two seconds; Q, then eight brushstrokes, gets you the rest. This is faster than any amount of tool-tuning, and it produces a better edge, because you are making decisions rather than adjusting a threshold that applies everywhere at once.

The two settings people never find

Double-click the Quick Mask button at the bottom of the toolbar and a dialogue appears with two choices worth knowing.

Colour Indicates: Masked Areas or Selected Areas. The default is masked, meaning red is what is not selected. If your painting seems inverted, this is why. Flip it to match how you think.

Opacity and colour of the overlay. The default is 50 per cent red. When you are masking something red, that is useless. Change it to green or blue for that job. This takes five seconds and saves a lot of squinting.

The brush settings that do the work

Masking is painting, so the brush settings are the technique.

• Hardness. A hard edge (100 per cent) for a manufactured object. Soft (0 to 20 per cent) for shadows, hair, out-of-focus regions. A soft brush is a feather you control per stroke.
• Opacity at 20 to 40 per cent, built up in passes. One stroke at full opacity is a decision you cannot modulate; four at 25 per cent lets you stop where it looks right.
• Flow is different from opacity: opacity caps the maximum a single stroke reaches, flow controls how quickly each dab builds towards it. Low flow with high opacity is the airbrush feel, best for shadow work.
• Size with [ and ]. Constantly. A mask painted with one brush size is always visible as a mask.

Refining an existing mask with paint

The same idea applies to a mask that already exists, without Quick Mask at all: click the mask thumbnail and paint on it directly. Alt/Option-click it first to see the mask on its own in black and white, which is the only way to find the ragged edges you cannot see against the picture.

Three repairs that come up constantly:

A halo of old background. Paint black along the edge with a small soft brush at low opacity, following the outline. Two passes usually removes it.

A hole in the middle. Paint white with a large hard brush. The interior of a subject has no edge to preserve, so precision there is wasted effort.

A too-hard transition. Select the mask, apply a Gaussian blur of 0.5 to 1 pixel, then a Levels move on the mask to pull the blurred edge back towards where it was. Blurring alone moves the edge outward slightly; the Levels correction is what keeps it in place.

Making a mask out of something that is not a selection

Because a mask is just a greyscale image, anything greyscale can become one. Copy a photograph, paste it into a mask, and the photograph’s tones now control visibility. Paste a scanned ink texture in and you get a grungy torn-edge effect with no brushwork at all. Paste a gradient and you have a graduated filter.

Alt/Option-click a mask thumbnail to enter it, then Ctrl/Cmd+V. This is the move behind most “how did they do that edge” questions.

The honest limits

Painting a selection is precise and it is slow. A complicated cut-out done entirely by hand at 100 per cent zoom is an hour. Use the automatic tools for the bulk and paint the fifteen per cent they got wrong.

It is also resolution-dependent in a way that catches people out. A mask painted while zoomed to 200 per cent can look crude at fit-to-screen, and a mask that looks perfect at fit-to-screen can be visibly blocky at 100 per cent. Check at both, and check the finished composite at the size it will be seen.

And hand-painting cannot create partial transparency that reflects reality. You can paint a 50 per cent grey along a hair edge, and it is your estimate of the coverage rather than a measurement of it. For fine edges, the matting methods later in this block do better, because they solve for the coverage from the pixel values instead of guessing it.