{"course":{"slug":"machine-learning-foundations","title":"Machine Learning, Foundations","url":"https://addaly.com/learn/machine-learning-foundations","level":"intermediate","track":"start"},"lesson":{"slug":"convolutions","title":"Convolution: the same detector, everywhere in the image","url":"https://addaly.com/learn/machine-learning-foundations/convolutions","position":80,"of":92,"minutes":9,"answer":"A convolution applies the same small filter at every position, so it learns a pattern once instead of separately per location.","keyIdea":"A convolution applies the same small filter at every position, so it learns a pattern once instead of separately per location, which cuts parameters by orders of magnitude and builds a hierarchy from edges to objects as depth increases."},"body":{"format":"markdown","text":"## Why a fully connected layer fails on images\n\nA 224 by 224 colour image is 150,528 numbers. A fully connected layer with 1,000 units needs 150 million weights for the first layer alone. That is unaffordable, and worse, it is wasteful in a specific way: a cat detector learned for the top-left corner would have to be learned again, from separate examples, for every other position.\n\nThe fully connected layer knows nothing about the fact that neighbouring pixels are related and that a pattern means the same thing wherever it appears.\n\n## What a convolution does instead\n\nSlide a small window (typically 3 by 3) across the image. At each position, multiply the window's values by a small set of learned weights and sum. That set of weights is a **filter** or kernel.\n\n```python\nnn.Conv2d(in_channels=3, out_channels=64, kernel_size=3, padding=1)\n```\n\nThis layer has 64 filters, each 3 by 3 across 3 input channels: `64 * 3 * 3 * 3 + 64 = 1,792` parameters. Compare that with 150 million.\n\nTwo ideas make the saving possible.\n\n**Local connectivity.** Each output depends on a 3 by 3 patch, not the whole image. Pixels far apart are combined only in later layers, and only after the nearby structure has been summarised.\n\n**Weight sharing.** The *same* filter is applied at every position. Learn an edge detector once and it works everywhere. This is what gives convolutional networks their translation equivariance: move the object in the image and the response moves with it.\n\n## What the filters learn\n\nThe learned filters are inspectable, and what they contain is consistent across networks and datasets.\n\n- **Layer 1**: edges at various orientations, colour blobs. These look strikingly like Gabor filters, which is what early visual cortex is understood to compute: a convergence nobody designed.\n- **Layer 2 to 3**: corners, curves, simple textures.\n- **Middle layers**: repeating patterns, object parts: a wheel, an eye, a doorway.\n- **Late layers**: whole objects and scene types.\n\nEach layer combines the previous layer's pieces. This is the compositional hierarchy from the depth lesson, made visible, and it is the reason transfer learning works so well in vision: the early layers are learning something general about images rather than something specific to the training task.\n\n## The receptive field\n\nOne 3 by 3 filter sees 3 pixels. Stack two and the second sees a 5 by 5 region of the original. Stack three and it is 7 by 7. Add pooling or a stride and the region grows much faster.\n\nThis is why depth matters in vision. A network needs a receptive field covering enough of the image to recognise a whole object, and stacking small filters is a cheaper way to get there than using one large one: two 3 by 3 layers have 18 weights per channel pair and cover the same region as a single 5 by 5 with 25, while including an extra non-linearity.\n\n## Pooling, and its decline\n\nMax pooling takes the maximum over each 2 by 2 block, halving the spatial dimensions. It reduces computation and gives a little tolerance to small shifts.\n\nModern architectures increasingly use strided convolutions instead (a convolution that moves 2 pixels at a time) which achieves the same downsampling with learned weights rather than a fixed rule. Pooling is not gone, and it is no longer automatic.\n\n## Where the field went\n\nTwo developments worth knowing so the picture is current.\n\n**Depthwise separable convolutions** split a convolution into a per-channel spatial step and a 1 by 1 mixing step, cutting the parameter count by roughly a factor of 8 to 9 with little accuracy loss. This is what makes MobileNet and its relatives run on a phone, and it is the reason on-device vision is practical at all.\n\n**Vision transformers** cut an image into patches and treat them as a sequence, with no convolution at all. They beat convolutional networks at very large data scales and lose to them at small ones, because the convolution's assumptions (locality and translation equivariance) are genuine prior knowledge that a transformer must learn from data instead. Hybrid designs now dominate.\n\nThat trade is a good general lesson. An architecture with built-in assumptions needs less data when the assumptions hold, and hits a ceiling the more flexible one does not.\n\n## What you will actually do\n\nAlmost nobody trains a vision network from scratch. You take a pretrained one and adapt it, which is the last lesson in this module. Free pretrained weights for hundreds of architectures are in `timm` and `torchvision`, and a fine-tuned ResNet on a few hundred images will beat anything you build from nothing on a free GPU."},"check":{"question":"Why do modern architectures prefer stacking two 3x3 convolutions over a single 5x5 convolution, given both cover the same receptive field?","options":[{"text":"The 3x3 filters can be applied in parallel, whereas a 5x5 must be computed sequentially.","correct":false,"tempting":"Both are parallel operations on modern hardware; stacked layers are in fact sequential with respect to each other."},{"text":"Two 3x3 layers use fewer weights per channel pair and insert an extra non-linearity between them, giving more expressive power for less parameter cost.","correct":true},{"text":"A 5x5 filter cannot be initialised stably, since its fan-in is too large for standard schemes.","correct":false,"tempting":"Initialisation schemes account for fan-in directly, so a 5x5 filter initialises perfectly well."},{"text":"Smaller filters produce a larger receptive field, which is what deep vision networks need.","correct":false,"tempting":"The premise states the receptive fields are the same, so a size advantage is not what distinguishes them."}]},"prev":{"slug":"regularising-a-network","title":"Keeping a large network from memorising","url":"https://addaly.com/learn/machine-learning-foundations/regularising-a-network"},"next":{"slug":"from-sequences-to-attention","title":"Sequences: from recurrence to attention","url":"https://addaly.com/learn/machine-learning-foundations/from-sequences-to-attention"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Convolution: the same detector, everywhere in the image', in Machine Learning, Foundations. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/machine-learning-foundations/convolutions (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Convolution: the same detector, everywhere in the image","courseTitle":"Machine Learning, Foundations","module":{"index":8,"slug":"neural-networks","title":"Neural networks, built from what you already know"},"answer":null,"keyIdea":"A convolution applies the same small filter at every position, so it learns a pattern once instead of separately per location, which cuts parameters by orders of magnitude and builds a hierarchy from edges to objects as depth increases.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"why-a-fully-connected-layer-fails-on-images","runs":[{"text":"Why a fully connected layer fails on images"}]},{"t":"p","runs":[{"text":"A 224 by 224 colour image is 150,528 numbers. A fully connected layer with 1,000 units needs 150 million weights for the first layer alone. That is unaffordable, and worse, it is wasteful in a specific way: a cat detector learned for the top-left corner would have to be learned again, from separate examples, for every other position."}]},{"t":"p","runs":[{"text":"The fully connected layer knows nothing about the fact that neighbouring pixels are related and that a pattern means the same thing wherever it appears."}]},{"t":"h2","id":"what-a-convolution-does-instead","runs":[{"text":"What a convolution does instead"}]},{"t":"p","runs":[{"text":"Slide a small window (typically 3 by 3) across the image. At each position, multiply the window’s values by a small set of learned weights and sum. That set of weights is a "},{"text":"filter","b":true},{"text":" or kernel."}]},{"t":"code","lang":"python","label":"Python","text":"nn.Conv2d(in_channels=3, out_channels=64, kernel_size=3, padding=1)"},{"t":"p","runs":[{"text":"This layer has 64 filters, each 3 by 3 across 3 input channels: "},{"text":"64 * 3 * 3 * 3 + 64 = 1,792","code":true},{"text":" parameters. Compare that with 150 million."}]},{"t":"p","runs":[{"text":"Two ideas make the saving possible."}]},{"t":"p","runs":[{"text":"Local connectivity.","b":true},{"text":" Each output depends on a 3 by 3 patch, not the whole image. Pixels far apart are combined only in later layers, and only after the nearby structure has been summarised."}]},{"t":"p","runs":[{"text":"Weight sharing.","b":true},{"text":" The "},{"text":"same","i":true},{"text":" filter is applied at every position. Learn an edge detector once and it works everywhere. This is what gives convolutional networks their translation equivariance: move the object in the image and the response moves with it."}]},{"t":"h2","id":"what-the-filters-learn","runs":[{"text":"What the filters learn"}]},{"t":"p","runs":[{"text":"The learned filters are inspectable, and what they contain is consistent across networks and datasets."}]},{"t":"ul","items":[[{"text":"Layer 1","b":true},{"text":": edges at various orientations, colour blobs. These look strikingly like Gabor filters, which is what early visual cortex is understood to compute: a convergence nobody designed."}],[{"text":"Layer 2 to 3","b":true},{"text":": corners, curves, simple textures."}],[{"text":"Middle layers","b":true},{"text":": repeating patterns, object parts: a wheel, an eye, a doorway."}],[{"text":"Late layers","b":true},{"text":": whole objects and scene types."}]]},{"t":"p","runs":[{"text":"Each layer combines the previous layer’s pieces. This is the compositional hierarchy from the depth lesson, made visible, and it is the reason transfer learning works so well in vision: the early layers are learning something general about images rather than something specific to the training task."}]},{"t":"h2","id":"the-receptive-field","runs":[{"text":"The receptive field"}]},{"t":"p","runs":[{"text":"One 3 by 3 filter sees 3 pixels. Stack two and the second sees a 5 by 5 region of the original. Stack three and it is 7 by 7. Add pooling or a stride and the region grows much faster."}]},{"t":"p","runs":[{"text":"This is why depth matters in vision. A network needs a receptive field covering enough of the image to recognise a whole object, and stacking small filters is a cheaper way to get there than using one large one: two 3 by 3 layers have 18 weights per channel pair and cover the same region as a single 5 by 5 with 25, while including an extra non-linearity."}]},{"t":"h2","id":"pooling-and-its-decline","runs":[{"text":"Pooling, and its decline"}]},{"t":"p","runs":[{"text":"Max pooling takes the maximum over each 2 by 2 block, halving the spatial dimensions. It reduces computation and gives a little tolerance to small shifts."}]},{"t":"p","runs":[{"text":"Modern architectures increasingly use strided convolutions instead (a convolution that moves 2 pixels at a time) which achieves the same downsampling with learned weights rather than a fixed rule. Pooling is not gone, and it is no longer automatic."}]},{"t":"h2","id":"where-the-field-went","runs":[{"text":"Where the field went"}]},{"t":"p","runs":[{"text":"Two developments worth knowing so the picture is current."}]},{"t":"p","runs":[{"text":"Depthwise separable convolutions","b":true},{"text":" split a convolution into a per-channel spatial step and a 1 by 1 mixing step, cutting the parameter count by roughly a factor of 8 to 9 with little accuracy loss. This is what makes MobileNet and its relatives run on a phone, and it is the reason on-device vision is practical at all."}]},{"t":"p","runs":[{"text":"Vision transformers","b":true},{"text":" cut an image into patches and treat them as a sequence, with no convolution at all. They beat convolutional networks at very large data scales and lose to them at small ones, because the convolution’s assumptions (locality and translation equivariance) are genuine prior knowledge that a transformer must learn from data instead. Hybrid designs now dominate."}]},{"t":"p","runs":[{"text":"That trade is a good general lesson. An architecture with built-in assumptions needs less data when the assumptions hold, and hits a ceiling the more flexible one does not."}]},{"t":"h2","id":"what-you-will-actually-do","runs":[{"text":"What you will actually do"}]},{"t":"p","runs":[{"text":"Almost nobody trains a vision network from scratch. You take a pretrained one and adapt it, which is the last lesson in this module. Free pretrained weights for hundreds of architectures are in "},{"text":"timm","code":true},{"text":" and "},{"text":"torchvision","code":true},{"text":", and a fine-tuned ResNet on a few hundred images will beat anything you build from nothing on a free GPU."}]}],"check":{"question":[{"text":"Why do modern architectures prefer stacking two 3x3 convolutions over a single 5x5 convolution, given both cover the same receptive field?"}],"options":[{"text":[{"text":"The 3x3 filters can be applied in parallel, whereas a 5x5 must be computed sequentially."}],"correct":false,"tempting":[{"text":"Both are parallel operations on modern hardware; stacked layers are in fact sequential with respect to each other."}]},{"text":[{"text":"Two 3x3 layers use fewer weights per channel pair and insert an extra non-linearity between them, giving more expressive power for less parameter cost."}],"correct":true,"tempting":null},{"text":[{"text":"A 5x5 filter cannot be initialised stably, since its fan-in is too large for standard schemes."}],"correct":false,"tempting":[{"text":"Initialisation schemes account for fan-in directly, so a 5x5 filter initialises perfectly well."}]},{"text":[{"text":"Smaller filters produce a larger receptive field, which is what deep vision networks need."}],"correct":false,"tempting":[{"text":"The premise states the receptive fields are the same, so a size advantage is not what distinguishes them."}]}],"order":[2,3,1,0],"why":null},"prev":{"slug":"regularising-a-network","title":"Keeping a large network from memorising","minutes":8},"next":{"slug":"from-sequences-to-attention","title":"Sequences: from recurrence to attention","minutes":9}}}