Convolutions: Sliding Pattern Detectors

Why dense layers fail at vision

Feed a 1-megapixel photo to a fully-connected layer of 1,000 neurons: 3 billion weights in the first layer alone. Worse, it has no idea that pixels have neighbors — shift the cat two pixels right and, to a dense layer, it's an utterly new input. Images have structure a good architecture should exploit: locality (nearby pixels relate) and translation invariance (a cat is a cat anywhere in the frame). Convolutional neural networks (CNNs) bake both in.

The convolution operation

A filter (kernel) is a tiny grid of weights, say 3×3. Slide it across the image; at each position, compute the dot product between the filter and the patch of pixels under it. The result is a feature map: a new image whose brightness at each location says how strongly the pattern appears there.

A filter with negative weights on the left column and positive on the right lights up wherever the image has a vertical edge. Other filters detect horizontal edges, corners, blobs, textures. Classic image processing hand-crafted such filters for decades. The CNN revolution: the filter weights are learned by backpropagation — the network invents whatever detectors minimize the loss.

Two structural superpowers:

• Weight sharing — one 3×3 filter = 9 weights, reused at every image position. Compare 3 billion. Fewer parameters = less overfitting + built-in translation invariance: the same detector runs everywhere, so a cat in any corner triggers it.
• Locality — each output looks at a small patch, matching how image structure actually works.

Assembling a CNN

A convolutional layer learns a bank of filters (e.g. 64), producing 64 feature maps ("channels"). Practical vocabulary: stride (slide step — stride 2 halves resolution), padding (add zeros at borders to control output size), 1×1 convolutions (mix channels without looking at neighbors).

Pooling shrinks maps by summarizing windows — max-pooling keeps the strongest response in each 2×2 block, downsampling the image while retaining "the pattern occurred here-ish". Modern nets often use strided convolutions instead; same effect, learnable.

The canonical stack alternates convolution → nonlinearity → downsample, while the channel count grows:

INPUT → [CONV+ReLU ×2 → POOL] × N → FLATTEN → DENSE → SOFTMAX

Spatial size shrinks, semantic depth grows: exactly the hierarchy from the "layers" lesson — edges → textures → parts → objects — but now you know the mechanism that learns it. Visualizing trained filters shows layer-1 edge detectors so consistently that they resemble the receptive fields neuroscientists measure in the visual cortex.

Key takeaways

• Convolution = a small learned pattern detector slid over the image; output maps say "pattern found here".
• Weight sharing + locality slash parameters and grant translation invariance — the right inductive bias for images.
• CNN = conv/ReLU/downsample stacks: resolution falls, meaning rises.