{"course":{"slug":"motion-and-audio","title":"Motion, Sound and Music","url":"https://addaly.com/learn/motion-and-audio","level":"intermediate","track":"create"},"lesson":{"slug":"tracking","title":"What a tracker can and cannot see","url":"https://addaly.com/learn/motion-and-audio/tracking","position":34,"of":72,"minutes":10,"answer":"A tracker correlates a patch of pixels against a search region.","keyIdea":"A tracker correlates a patch of pixels against a search region, so a feature on a straight edge is unsolvable along that edge (the aperture problem) and every good feature is one with contrast in two directions."},"body":{"format":"markdown","text":"## What a tracker is actually doing\n\nA 2D point tracker does one thing per frame: it takes a small rectangle of pixels from the previous frame (the pattern) and searches a larger rectangle in the current frame for the position where those pixels match best. It reports that position, usually to a fraction of a pixel by fitting a curve to the correlation scores around the best whole-pixel match.\n\nThat is the whole mechanism. Every tracking failure follows from it, and once you can name the mechanism you stop guessing at settings.\n\n## The aperture problem, which explains most failures\n\nPut a tracker on a long straight edge: the top of a table, the join between two walls, a cable against the sky. The track slides along the edge and the result is useless.\n\nThe reason is geometric. Through a small window onto a straight edge, motion *along* the edge produces no change in the pixels at all. The pattern at position x looks identical to the pattern at position x+5. The tracker can measure the component of motion perpendicular to the edge and has no information whatsoever about the component parallel to it. This is the aperture problem, and it is not a limitation of the software.\n\nThe consequence is the single most useful rule in tracking: **a good feature has contrast in two directions.** A corner. A crossing. A bolt head. A high-contrast blemish. Not an edge, not a smooth gradient, not a repeating texture like brickwork: repeats give the search several equally good answers and the track jumps between them.\n\nBlender's tracker and most others can show you a correlation score per frame. A feature that is trackable holds above 0.9; one that drifts along an edge falls off steadily and tells you before the result is visibly wrong.\n\n## Pattern size and search size\n\nTwo rectangles, two different jobs.\n\nThe **pattern** is what is being matched. Make it large enough to contain a distinctive feature and small enough that everything inside it moves together. If your pattern spans two objects at different depths, they move by different amounts and no single position matches both.\n\nThe **search area** is where the tracker looks. It must be large enough to contain the frame-to-frame movement and no larger: a bigger search area is slower and offers more chances to find a false match. If the camera whips, the movement between frames can be a hundred pixels and a default search area will lose the feature.\n\nWhen a track fails on a fast move, the cause is nearly always the search area, not the pattern.\n\n## The four things that break a track\n\n**Motion blur.** A smeared feature has no sharp structure to correlate against. Trackers commonly survive mild blur and fail on heavy blur, and the usual answer is to track a different, larger feature that stays distinguishable when smeared.\n\n**Occlusion.** Something passes in front. Every tracker offers a way to keyframe the track manually through the occlusion and resume; on a short occlusion, interpolating between the position before and after is usually accurate enough.\n\n**Lighting change.** The pattern is matched on pixel values, so a feature that moves through a shadow changes its values. Most trackers can normalise for brightness, which handles a gradual change and not a hard shadow edge.\n\n**Parallax inside the pattern.** A feature near a depth discontinuity (a corner of a building against a distant background) changes its surroundings as the camera moves. Shrink the pattern until it contains only the near object.\n\n## Using a track\n\nOnce you have a path, there are three standard applications.\n\n- **Stabilise.** Apply the inverse of the track to the footage. This crops, because the frame is being moved around inside itself, so a stabilised shot is always smaller than its source: typically 5 to 15 per cent. Decide this before framing the shot, not after.\n- **Match-move.** Apply the track to an inserted element so it appears attached. One point gives position; two points give position, rotation and scale; four points give a full perspective corner pin.\n- **Drive an effect.** A blur, a censor patch, a colour correction that follows a moving object.\n\n## Planar over point, when you can\n\nFor any surface that is flat and stays flat, a planar track is more robust than four point tracks, because it uses thousands of pixels rather than four small patches and can tolerate parts of the surface being occluded or badly lit. A phone screen, a poster, a wall, a face's forehead: all planar enough. This is the practical reason planar tracking took over so much of the work that used to be point tracking.\n\n## The free path\n\n**Blender**'s motion tracking is a full implementation (2D tracks with adjustable pattern and search, correlation display, plane tracks, and a solver) and it is free with no watermark and no resolution cap. It is the single best free tool in this module. **Fusion** in DaVinci Resolve's free edition has a tracker and a planar tracker. **Kdenlive** and **Shotcut** both expose simple motion tracking for effects. Mocha is the paid standard and worth naming on a CV.\n\n## The limitation\n\nA tracker reports where a pattern of pixels went. It does not know what an object is. It will happily track a reflection, a shadow, or a specular highlight that is sliding across a surface rather than attached to it, and a highlight is a superb-looking track that is physically wrong, because its position depends on the light and the camera rather than on the object.\n\nThe result looks convincing in the tracker's own display and slips visibly once you attach something to it. The defence is to check what you are tracking with your eyes before you press the button, and to test the result by attaching a crosshair and playing the shot at full speed."},"check":{"question":"A track placed on the long horizontal join between two wall panels drifts sideways over the shot, although the correlation score stays high. What is happening?","options":[{"text":"The pattern box is larger than the search box, so the tracker is matching a subset of the pattern at several positions and choosing whichever scores highest by chance.","correct":false,"tempting":"A pattern larger than the search area is a configuration error that causes failure or jumping, not steady drift with a high score."},{"text":"Motion blur along the direction of camera movement has smeared the join, so the tracker is locking onto the blur's leading edge, which moves faster than the feature itself.","correct":false,"tempting":"Blur degrades correlation scores; the high score here indicates the tracker is matching well, just without lateral information."},{"text":"Through a small window onto a straight edge, motion along the edge produces no change in the pixels, so the tracker has no information about that component at all.","correct":true},{"text":"The join is a low-frequency feature, and sub-pixel interpolation of the correlation surface is only accurate for high-frequency detail, so the reported position accumulates a rounding error each frame.","correct":false,"tempting":"Sub-pixel fitting is less precise on soft features, but that produces jitter rather than a consistent one-directional slide."}]},"prev":{"slug":"rotoscoping","title":"Drawing the matte by hand","url":"https://addaly.com/learn/motion-and-audio/rotoscoping"},"next":{"slug":"camera-solve-and-parallax","title":"Solving for a camera, and when you cannot","url":"https://addaly.com/learn/motion-and-audio/camera-solve-and-parallax"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What a tracker can and cannot see', in Motion, Sound and Music. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/motion-and-audio/tracking (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What a tracker can and cannot see","courseTitle":"Motion, Sound and Music","module":{"index":4,"slug":"putting-two-images-together","title":"Putting two images together"},"answer":null,"keyIdea":"A tracker correlates a patch of pixels against a search region, so a feature on a straight edge is unsolvable along that edge (the aperture problem) and every good feature is one with contrast in two directions.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"what-a-tracker-is-actually-doing","runs":[{"text":"What a tracker is actually doing"}]},{"t":"p","runs":[{"text":"A 2D point tracker does one thing per frame: it takes a small rectangle of pixels from the previous frame (the pattern) and searches a larger rectangle in the current frame for the position where those pixels match best. It reports that position, usually to a fraction of a pixel by fitting a curve to the correlation scores around the best whole-pixel match."}]},{"t":"p","runs":[{"text":"That is the whole mechanism. Every tracking failure follows from it, and once you can name the mechanism you stop guessing at settings."}]},{"t":"h2","id":"the-aperture-problem-which-explains-most-failures","runs":[{"text":"The aperture problem, which explains most failures"}]},{"t":"p","runs":[{"text":"Put a tracker on a long straight edge: the top of a table, the join between two walls, a cable against the sky. The track slides along the edge and the result is useless."}]},{"t":"p","runs":[{"text":"The reason is geometric. Through a small window onto a straight edge, motion "},{"text":"along","i":true},{"text":" the edge produces no change in the pixels at all. The pattern at position x looks identical to the pattern at position x+5. The tracker can measure the component of motion perpendicular to the edge and has no information whatsoever about the component parallel to it. This is the aperture problem, and it is not a limitation of the software."}]},{"t":"p","runs":[{"text":"The consequence is the single most useful rule in tracking: "},{"text":"a good feature has contrast in two directions.","b":true},{"text":" A corner. A crossing. A bolt head. A high-contrast blemish. Not an edge, not a smooth gradient, not a repeating texture like brickwork: repeats give the search several equally good answers and the track jumps between them."}]},{"t":"p","runs":[{"text":"Blender’s tracker and most others can show you a correlation score per frame. A feature that is trackable holds above 0.9; one that drifts along an edge falls off steadily and tells you before the result is visibly wrong."}]},{"t":"h2","id":"pattern-size-and-search-size","runs":[{"text":"Pattern size and search size"}]},{"t":"p","runs":[{"text":"Two rectangles, two different jobs."}]},{"t":"p","runs":[{"text":"The "},{"text":"pattern","b":true},{"text":" is what is being matched. Make it large enough to contain a distinctive feature and small enough that everything inside it moves together. If your pattern spans two objects at different depths, they move by different amounts and no single position matches both."}]},{"t":"p","runs":[{"text":"The "},{"text":"search area","b":true},{"text":" is where the tracker looks. It must be large enough to contain the frame-to-frame movement and no larger: a bigger search area is slower and offers more chances to find a false match. If the camera whips, the movement between frames can be a hundred pixels and a default search area will lose the feature."}]},{"t":"p","runs":[{"text":"When a track fails on a fast move, the cause is nearly always the search area, not the pattern."}]},{"t":"h2","id":"the-four-things-that-break-a-track","runs":[{"text":"The four things that break a track"}]},{"t":"p","runs":[{"text":"Motion blur.","b":true},{"text":" A smeared feature has no sharp structure to correlate against. Trackers commonly survive mild blur and fail on heavy blur, and the usual answer is to track a different, larger feature that stays distinguishable when smeared."}]},{"t":"p","runs":[{"text":"Occlusion.","b":true},{"text":" Something passes in front. Every tracker offers a way to keyframe the track manually through the occlusion and resume; on a short occlusion, interpolating between the position before and after is usually accurate enough."}]},{"t":"p","runs":[{"text":"Lighting change.","b":true},{"text":" The pattern is matched on pixel values, so a feature that moves through a shadow changes its values. Most trackers can normalise for brightness, which handles a gradual change and not a hard shadow edge."}]},{"t":"p","runs":[{"text":"Parallax inside the pattern.","b":true},{"text":" A feature near a depth discontinuity (a corner of a building against a distant background) changes its surroundings as the camera moves. Shrink the pattern until it contains only the near object."}]},{"t":"h2","id":"using-a-track","runs":[{"text":"Using a track"}]},{"t":"p","runs":[{"text":"Once you have a path, there are three standard applications."}]},{"t":"ul","items":[[{"text":"Stabilise.","b":true},{"text":" Apply the inverse of the track to the footage. This crops, because the frame is being moved around inside itself, so a stabilised shot is always smaller than its source: typically 5 to 15 per cent. Decide this before framing the shot, not after."}],[{"text":"Match-move.","b":true},{"text":" Apply the track to an inserted element so it appears attached. One point gives position; two points give position, rotation and scale; four points give a full perspective corner pin."}],[{"text":"Drive an effect.","b":true},{"text":" A blur, a censor patch, a colour correction that follows a moving object."}]]},{"t":"h2","id":"planar-over-point-when-you-can","runs":[{"text":"Planar over point, when you can"}]},{"t":"p","runs":[{"text":"For any surface that is flat and stays flat, a planar track is more robust than four point tracks, because it uses thousands of pixels rather than four small patches and can tolerate parts of the surface being occluded or badly lit. A phone screen, a poster, a wall, a face’s forehead: all planar enough. This is the practical reason planar tracking took over so much of the work that used to be point tracking."}]},{"t":"h2","id":"the-free-path","runs":[{"text":"The free path"}],"role":"free"},{"t":"p","runs":[{"text":"Blender","b":true},{"text":"’s motion tracking is a full implementation (2D tracks with adjustable pattern and search, correlation display, plane tracks, and a solver) and it is free with no watermark and no resolution cap. It is the single best free tool in this module. "},{"text":"Fusion","b":true},{"text":" in DaVinci Resolve’s free edition has a tracker and a planar tracker. "},{"text":"Kdenlive","b":true},{"text":" and "},{"text":"Shotcut","b":true},{"text":" both expose simple motion tracking for effects. Mocha is the paid standard and worth naming on a CV."}]},{"t":"h2","id":"the-limitation","runs":[{"text":"The limitation"}],"role":"limit"},{"t":"p","runs":[{"text":"A tracker reports where a pattern of pixels went. It does not know what an object is. It will happily track a reflection, a shadow, or a specular highlight that is sliding across a surface rather than attached to it, and a highlight is a superb-looking track that is physically wrong, because its position depends on the light and the camera rather than on the object."}]},{"t":"p","runs":[{"text":"The result looks convincing in the tracker’s own display and slips visibly once you attach something to it. The defence is to check what you are tracking with your eyes before you press the button, and to test the result by attaching a crosshair and playing the shot at full speed."}]}],"check":{"question":[{"text":"A track placed on the long horizontal join between two wall panels drifts sideways over the shot, although the correlation score stays high. What is happening?"}],"options":[{"text":[{"text":"The pattern box is larger than the search box, so the tracker is matching a subset of the pattern at several positions and choosing whichever scores highest by chance."}],"correct":false,"tempting":[{"text":"A pattern larger than the search area is a configuration error that causes failure or jumping, not steady drift with a high score."}]},{"text":[{"text":"Motion blur along the direction of camera movement has smeared the join, so the tracker is locking onto the blur’s leading edge, which moves faster than the feature itself."}],"correct":false,"tempting":[{"text":"Blur degrades correlation scores; the high score here indicates the tracker is matching well, just without lateral information."}]},{"text":[{"text":"Through a small window onto a straight edge, motion along the edge produces no change in the pixels, so the tracker has no information about that component at all."}],"correct":true,"tempting":null},{"text":[{"text":"The join is a low-frequency feature, and sub-pixel interpolation of the correlation surface is only accurate for high-frequency detail, so the reported position accumulates a rounding error each frame."}],"correct":false,"tempting":[{"text":"Sub-pixel fitting is less precise on soft features, but that produces jitter rather than a consistent one-directional slide."}]}],"order":[0,2,1,3],"why":null},"prev":{"slug":"rotoscoping","title":"Drawing the matte by hand","minutes":10},"next":{"slug":"camera-solve-and-parallax","title":"Solving for a camera, and when you cannot","minutes":11}}}