Eight by eight, and what the encoder threw away

The encoder’s four steps

JPEG has been the default photographic format since 1992 and the reason it works is worth twenty minutes of your life, because every artefact you will ever have to argue with a client about comes from one of four steps.

1. Convert to brightness and two colour differences. RGB becomes Y (luminance) plus Cb and Cr (blue-difference and red-difference). Your eye is far more sensitive to brightness detail than colour detail, and this split lets the encoder treat them differently.

2. Throw away colour resolution. In 4:2:0 subsampling, the near-universal default, the Cb and Cr channels are stored at half resolution in each direction, so one colour sample covers a 2×2 block of pixels. That is three quarters of the colour data discarded immediately, and on a face or a landscape you cannot see it. On a thin red line on white you can, which is the next section.

3. Split into 8×8 blocks and transform each one. Each block goes through a discrete cosine transform, which re-describes those 64 values as 64 coefficients: one average level plus 63 patterns of increasing fineness. Nothing is lost yet; this is a lossless re-description.

4. Quantise. Each coefficient is divided by a number from a quantisation table and rounded. The fine-detail coefficients get divided by large numbers, so most of them round to zero. This is the lossy step, and it is the only one. The quality slider does nothing except scale that table: quality 90 uses small divisors, quality 40 uses large ones.

The zeros then compress to almost nothing, which is where the file size comes from.

Reading an artefact backwards

Knowing the four steps means you can look at a bad file and say which step caused it.

Blocking: visible 8×8 squares, usually in sky or shadow. Quantisation was so coarse that only the block’s average level survived. Cause: quality set too low, or an image re-saved many times.

Ringing, also called mosquito noise: a shimmer of light and dark just outside a hard edge, most obvious around black text on white. The encoder is describing a step edge as a sum of smooth cosine waves; truncating that sum leaves overshoot on either side. This is a mathematical property of the transform, not a bug, and it is why JPEG is wrong for screenshots, diagrams and text.

Colour bleed and dirty fringes on saturated text: step 2. Red type on white is a hard edge in a channel that was stored at half resolution. Raising the quality slider does not fix it, because subsampling happens before quantisation. Some encoders let you force 4:4:4 (no subsampling); Photoshop does it automatically at quality 7 and above in Save for Web, and cjpeg -sample 1x1 does it explicitly. Or just use PNG or WebP, which is the real answer.

Generation loss: open a JPEG, make one small edit, save at quality 85, repeat. After 30 rounds the edges crawl and the flat areas terrace. What happens is that each decode lands on values that do not sit neatly on the new quantisation grid, so each re-encode rounds again. It does eventually converge to a stable state, but well below where it started. Re-saving without any edit in a tool that re-encodes still loses a little each time; re-saving with identical settings converges faster but never recovers.

Things you can do losslessly

Not every JPEG operation has to re-encode. jpegtran does several transformations by rearranging the coefficient blocks directly:

jpegtran -rotate 90 -perfect -copy all in.jpg > out.jpg
jpegtran -crop 800x600+16+16 in.jpg > cropped.jpg

Rotation is exact if the dimensions are multiples of 8 or 16, which -perfect enforces. Cropping is lossless only on block boundaries, so the offsets must be multiples of 8. Most phone gallery apps that offer “rotate” do this correctly; most desktop editors do not, and a rotate in an ordinary editor costs you a full generation.

Progressive JPEG is worth knowing: the same data reordered so a coarse version arrives first and refines. It is typically 2 to 10 per cent smaller than baseline and shows something useful earlier on a slow connection. There is no quality difference. Turn it on for anything going on the web.

MozJPEG, a free encoder from Mozilla, produces files roughly 5 to 15 per cent smaller than the standard library at matched visual quality by choosing better tables and trellis-optimising the coefficients. Squoosh in a browser uses it; cjpeg from the mozjpeg package uses it on the command line; it is free either way.

The honest limits

Quality numbers are not comparable between encoders. Quality 80 in libjpeg, in MozJPEG, in Photoshop’s Save As and in Photoshop’s Save for Web are four different things producing four different file sizes. Photoshop’s Save As runs 0 to 12 and its Save for Web runs 0 to 100 and they do not correspond. Never specify a quality number in a brief without naming the tool; specify a file size or a visual test instead.

Artefacts cannot be removed. There are deblocking filters and AI restoration models, and what they do is smooth or regenerate, not recover. The coefficients were rounded to zero; the information is gone. The only fix for a badly compressed file is a better source file.

And the whole thing is aimed at photographs of the real world. JPEG’s assumptions (smooth gradients, soft edges, colour that changes slowly) are true of a photograph and false of a screenshot, a chart, a line drawing or a logo. Using it on those is not a bad setting; it is the wrong tool.