CNNs — convolutions for vision

📅 Day 46. CNNs (Convolutional Neural Networks) revolutionised computer vision in 2012 and still dominate small-to-medium image tasks today.

🖼 The core operation — convolution.

A 2D convolution slides a small kernel (typically 3x3 or 5x5) across an image. At each position, it computes a dot product between the kernel's weights and the corresponding image patch. The output is a new 2D map of activations.

The kernel learns features — edges, corners, textures, then more abstract patterns in deeper layers (eyes, wheels, faces). Same kernel, applied everywhere — translation invariance built in.

🎯 Why CNNs beat MLPs on images:

→ Locality. Pixels near each other are related; the convolution exploits that. An MLP has to relearn this from scratch.

→ Translation invariance. A cat at the top-left of an image is still a cat at the bottom-right. Same kernel applied at every position handles it naturally. An MLP would need separate parameters for each position.

→ Parameter sharing. The same kernel weights are used across the entire image. A 3x3 kernel has 9 parameters (plus a bias). An MLP doing equivalent work would have millions.

→ Hierarchical features. Stack convolution layers; later layers see combinations of earlier features. Layer 1 learns edges; layer 2 learns corners and textures; layer 5 learns object parts; layer 10 learns objects.

🏗 The standard CNN architecture — alternating convolution + pooling layers, ending with one or two fully-connected layers for classification. Pooling (max or average) downsamples the feature maps, reducing computation and adding some translation invariance.

💡 Modern CNNs (ResNet, EfficientNet, ConvNeXt) refine this — skip connections, depthwise separable convolutions, channel attention. The core convolution operation is unchanged.

🚀 Convolutions for vision is still the right tool. Even in 2026 when Vision Transformers exist, ResNet wins on small datasets and inference cost.

🏗 ResNet (2015) is one of the most influential papers in deep learning. The reason it survived a decade — it solved a fundamental problem nothing else had.

🚨 The pre-ResNet problem. Adding more layers to a CNN made things WORSE, not better, past about 20 layers. Even with proper initialisation and ReLU activations, deep networks failed to train. Gradients vanished or exploded; layers stopped learning meaningfully.

This was counterintuitive — more parameters should mean more capacity. But empirically, deep networks underperformed shallow ones.

🎯 ResNet's insight — skip connections. Add the input back to the output of every block.

output = block(input) + input

Instead of computing block(input), compute block(input) + input. The block now learns the RESIDUAL (the difference between input and desired output) rather than the full transformation.

🌊 Why this works:

→ Gradient flow. The gradient flows directly through the skip connection during backprop. Even if block(input) has near-zero gradient, the +input term keeps things flowing. Networks 100+ layers deep become trainable.

→ Identity is easy. If a layer has nothing useful to add, it can output approximately zero (the residual is zero), and the input passes through unchanged. The network learns when to use a layer.

→ Implicit ensemble. With many skip connections, the network has many parallel paths. It behaves like an ensemble of shallower networks.

🏆 In 2026, ResNet-50 (50 layers) is still the default backbone for many vision tasks. Vision Transformers (ViT) match or beat it on big data; ResNet wins on small datasets and at inference cost (smaller, faster).

💡 The skip-connection idea spread far beyond ResNet. Transformers use it. Modern LLMs use it. U-Net for segmentation uses it. The +input pattern is one of deep learning's most reused ideas.

🚀 Two characters. Massive impact.

💻 The minimum viable CNN. 14 lines. Hits 99% accuracy on MNIST after 5 epochs. The skeleton of every computer-vision model you'll ever write.

Look at the snippet.

🏗 Class TinyCNN inherits from nn.Module. Layers defined in __init__ as nn.Sequential.

📦 First convolution block:
→ nn.Conv2d(1, 16, 3, padding=1) — 1 input channel (grayscale), 16 output channels (16 different kernels), 3x3 kernel size, padding=1 (keeps spatial size unchanged).
→ nn.ReLU() — activation.
→ nn.MaxPool2d(2) — downsample by 2x in each spatial dim. 28x28 image becomes 14x14.

📦 Second convolution block:
→ nn.Conv2d(16, 32, 3, padding=1) — 16 input channels, 32 output channels.
→ nn.ReLU().
→ nn.MaxPool2d(2) — 14x14 becomes 7x7.

🔄 Classifier:
→ nn.Flatten() — turn (32, 7, 7) tensor into (32 * 7 * 7,) vector. 1568 features.
→ nn.Linear(1568, 128) — fully connected layer to 128 hidden units.
→ nn.ReLU().
→ nn.Linear(128, n_classes) — output layer to 10 logits (one per digit).

No softmax — CrossEntropyLoss applies it internally for numerical stability.

📊 The shape progression — (B, 1, 28, 28) → (B, 16, 14, 14) → (B, 32, 7, 7) → (B, 1568) → (B, 128) → (B, 10).

B is batch size. Channels grow (1 → 16 → 32) as spatial size shrinks (28 → 14 → 7). The flatten + dense layers do the final classification.

🚀 This shape — alternating conv+pool, flatten, dense — is the basic CNN template. Larger models add more conv blocks, batch normalisation, dropout, residual connections. The skeleton is the same.

💡 14 lines. Real working CV model.

💡 Pro tip — for any computer vision task in 2026, start from a pretrained model. Training from scratch is almost always wrong.

🤔 Why? Two reasons.

📊 Pretrained models on ImageNet (or larger datasets) have learned useful features — edges, textures, shapes, common objects. These features TRANSFER to your task. Even if you're classifying medical images or industrial defects, the low-level features are similar.

⏱ Training from scratch on small datasets (under 100k images) usually leads to overfitting. The model has too many parameters relative to your training data; it memorises rather than learns.

Fine-tuning a pretrained model on your data — typically 10x faster training, 10-30% better accuracy on small datasets. Free lunch.

📦 The torchvision.models module has many pretrained options:

→ ResNet (resnet18, resnet50) — the workhorse. Pretrained on ImageNet (1.2M images, 1000 classes).

→ EfficientNet (efficientnet_b0 through b7) — better accuracy/cost tradeoff than ResNet. Use for production where inference cost matters.

→ ViT (vit_b_16) — Vision Transformers. Best accuracy on big data; slower inference. Use when you have lots of training data.

→ ConvNeXt — modern CNN with ViT-inspired changes. Strong all-around.

🎯 The fine-tuning recipe:

1️⃣ Load pretrained model. model = resnet50(weights='IMAGENET1K_V2').

2️⃣ Replace the final layer. Originally 1000 classes; you have N. model.fc = nn.Linear(model.fc.in_features, N).

3️⃣ Fine-tune. Train all layers with a small learning rate (1e-4 or smaller).

4️⃣ Optionally — freeze early layers, train only the last few. Faster, less risk of overfitting on tiny datasets.

💡 The transfer-learning lunch is free. Eat it.

🚀 Use Hugging Face's image models for even more options — DINOv2 for self-supervised features, CLIP for image-text alignment, SAM for segmentation. The library is large; the API is clean.

📅 End of Day 46.

✅ Recap:

🖼 Convolution = sliding dot product + parameter sharing. Locality, translation invariance, hierarchical features. The right tool for vision.

🌊 ResNet's skip connections (x + f(x)) unlocked deep CNNs. Networks 100+ layers deep became trainable. Pattern reused everywhere — transformers, U-Nets, modern LLMs.

💻 Tiny CNN for MNIST in 14 lines. Conv → ReLU → MaxPool → repeat → Flatten → Dense. The skeleton of every CV model.

📦 Always start from a pretrained model. Training from scratch on small data overfits. ResNet, EfficientNet, ViT — all pretrained on ImageNet, fine-tune for your task. 10-30% accuracy lift for free.

🧠 Reflection — vision is one of the most successful applications of deep learning. From AlexNet (2012) to today, the architecture has refined but the convolution operation is unchanged. The lesson — build on what works; don't rewrite the foundation when refinements suffice.

🚀 Tomorrow, Day 47 — RNNs and LSTMs. Mostly historical now (transformers won), but the foundation that made transformers necessary. The 'pack variable-length sequences' tip for production.

💼 We've covered the two main neural-network families — feedforward (MLPs, CNNs) and recurrent (tomorrow). Both are stepping stones to transformers (Day 53) which dominate modern AI.

👋 See you in the morning.