What you'll learn

Builds on Modules 1–3

≈6 h · 2 h video · 2 h reading · 2 h coding

Show MLP spatial limits (shifted inputs) vs a simple CNN
Implement convolutional layers and train a small CNN
Apply augmentation and dropout; run a brief augmentation ablation

Lecture 4 – Convolutional Neural Networks (CNNs)

DL4.2 – Coding up your first CNN

📚 Resources & Lecture Code

Course book · primary reading

Alternative reading from Dive into Deep Learning (D2L):

The Colab notebook includes the lecture code for Module 4 (CNNs and a first PyTorch CNN). Use it to reproduce the code demos from lecture.

Open Colab Notebook (shared for Modules 4–5)

🧠 Test your understanding

Try each question before revealing the answer — these mirror the ideas the module quiz checks.

Q1.For a convolution with kernel size

k

, stride

s

, padding

p

, and input size

n

, what is the output spatial size?

$\lfloor (n + 2p - k)/s \rfloor + 1$
$\lfloor (n - 2p + k)/s \rfloor + 1$
$\lfloor (n + p - k)/s \rfloor + 1$
$\lfloor (n + 2p + k)/s \rfloor + 1$

Show answer ▾

Answer: $\lfloor (n + 2p - k)/s \rfloor + 1$

This formula accounts for padding added on both sides (2p), the kernel size subtracted, division by stride, plus 1 for the initial position. It is fundamental for CNN design and appears in every architecture textbook.

Q2.What is the receptive field of a neuron in a CNN?

The number of parameters in its kernel
The region of the input that affects its output
The number of channels in its layer
The size of the feature map it produces

Show answer ▾

Answer: The region of the input that affects its output

Receptive field is a core concept: larger receptive fields in deeper layers allow networks to capture global context. This is crucial for understanding why stacking convolutions increases representational power.

Q3.What is the main purpose of pooling layers in CNNs?

To increase the number of parameters
To add non-linearity to the network
To reduce spatial dimensions and add local tolerance to small translations
To normalize the activations

Show answer ▾

Answer: To reduce spatial dimensions and add local tolerance to small translations

Pooling downsamples feature maps and can make a response insensitive to a shift that stays within the same pooling neighborhood. It does not make the complete CNN exactly translation-invariant: window boundaries, stride, padding, clipping, and later position-sensitive layers still matter.

Q4.How many parameters does a convolutional layer with 32 input channels, 64 output channels, and 3×3 kernels have (including bias)?

576
18,432
18,496
32,832

Show answer ▾

Answer: 18,496

Parameters = $(k_h \times k_w \times C_{in} \times C_{out}) + C_{out} = (3 \times 3 \times 32 \times 64) + 64 = 18,496$ . The bias term (one per output channel) is often overlooked but matters for accurate parameter counting.