Frames that stand alone, and frames that do not

Why scrubbing a phone video feels like wading

Drag the playhead through footage straight off a camera and it lags, sometimes by half a second. Drag through the same footage converted to ProRes and it responds instantly, from a file five times larger. Nothing about your machine changed. The difference is whether each frame can be decoded on its own.

Three kinds of frame

I-frame (intra-coded, also called a keyframe). Compressed on its own, like a JPEG. Everything needed to reconstruct it is in the frame.

P-frame (predicted). Stored as a set of instructions relative to an earlier frame: take that block, move it 14 pixels left and 3 up, and add this small correction. For typical footage a P-frame is a small fraction of an I-frame’s size.

B-frame (bidirectional). Predicted from frames both before and after it in display order. Even smaller. Because they reference a future frame, the frames have to be transmitted out of display order, which is why a video has a decode order and a presentation order that differ.

A GOP (group of pictures) is one I-frame and the dependent frames that follow it. A typical streaming encode uses a GOP of one to two seconds: at 25fps, an I-frame every 25 to 50 frames.

What follows for editing

To show frame 43 of a GOP starting at frame 1, the decoder must decode frames 1 through 43. There is no way around it; frame 43 is defined in terms of its predecessors. Scrub backwards and it is worse, because going back one frame means decoding forward from the keyframe again.

This is why editing “long-GOP” material is slow, and why the standard professional move is to transcode to an all-intra codec first. Every frame is independent, every seek is a single decode, and the price is file size.

The all-intra options, with the free ones named:

• ProRes (Apple) and DNxHR (Avid): the two paid-ecosystem standards, both readable and writable by ffmpeg and by the free DaVinci Resolve. ProRes 422 runs around 150 Mbit/s at 1080p25: roughly 20 GB an hour.
• FFV1: open source, mathematically lossless, and the format chosen by several national archives, including for preservation work at the Library of Congress. In a Matroska container with checksums per frame, it is the free archival answer.
• Ut Video and HuffYUV: free lossless intermediates, fast, larger than FFV1.
• MJPEG: every frame a JPEG. Old, inefficient, universally decodable, and still occasionally the pragmatic answer.

Keyframes and streaming

The GOP length is a trade-off with two visible consequences.

Seeking granularity. A player can only start at a keyframe. With a 10-second GOP, clicking the middle of a progress bar means the player decodes up to 10 seconds of video before it can show you anything, or shows you a frame up to 10 seconds away from where you clicked.

Adaptive streaming. HLS and DASH cut the video into segments and switch quality at segment boundaries, which must be keyframes. If your keyframe interval does not divide evenly into the segment length, the packager inserts extra keyframes or the segments drift.

The convention that avoids both problems: a keyframe every 2 seconds, with segments of 2, 4 or 6 seconds.

# 2-second keyframe interval at 25fps, no scene-cut keyframes
ffmpeg -i in.mov -c:v libx264 -g 50 -keyint_min 50 -sc_threshold 0 out.mp4

-sc_threshold 0 turns off automatic keyframes at scene cuts. For general viewing you want them on, because a cut is exactly where a fresh I-frame is cheap and useful. For adaptive streaming you want them off, so every keyframe is where the packager expects it.

Cutting without re-encoding

Because a GOP is a unit, you can trim a file at keyframe boundaries without touching the compressed data:

ffmpeg -ss 00:01:30 -i in.mp4 -t 00:00:45 -c copy out.mp4

-c copy means no decode and no encode: it is nearly instantaneous and there is no generational loss at all. The catch is that the cut lands on the nearest keyframe, so your out point can be up to a GOP away from where you asked. For a rough trim before uploading, this is the right tool and almost nobody uses it.

Where long-GOP wins

Nothing above says long-GOP is bad. For delivery it is dramatically more efficient: the whole reason streaming is possible. A 45-minute programme at 5 Mbit/s H.264 is about 1.7 GB; the same programme in ProRes 422 is about 50 GB.

The rule is the point in the chain. Long-GOP for delivery, all-intra for work. Acquisition sits in between and increasingly uses long-GOP for capacity, which is why the transcode step exists.

Proxies: the free middle path

You do not have to transcode to a huge intermediate to edit comfortably. A proxy is a small, low-resolution, all-intra copy that the editor uses for playback while the timeline still refers to the original for export.

ffmpeg -i big.mp4 -vf scale=-2:540 -c:v dnxhd -profile:v dnxhr_lb \
  -c:a pcm_s16le proxy.mov

Every serious editor supports this: DaVinci Resolve, Kdenlive and Shotcut all have proxy workflows in their free versions. It is the answer to “my laptop cannot edit 4K”, and it is far more effective than any hardware upgrade at the same cost.

The free path

ffmpeg encodes and decodes every format named here, including ProRes and DNxHR. DaVinci Resolve’s free edition has a full proxy and optimised-media workflow. Kdenlive and Shotcut both generate proxies with one setting. FFV1 and Matroska are both open standards with no licensing questions, which is precisely why archives chose them.

The limitation

All-intra editing costs storage at a rate that surprises people. An hour of ProRes 422 HQ at 1080p is roughly 30 GB; at 4K it is over 100 GB. A week of shooting fills a drive, and the backup of that drive fills another.

There is also no single intermediate everyone agrees on. ProRes is effectively a standard and is Apple’s; DNxHR is Avid’s; FFV1 is open and slower to encode and not accepted by many facilities. Choosing one is a workflow decision with political and practical edges, and anybody who tells you there is an obvious answer works somewhere that already made it.