6  When It Fails: Generalization in Pictures, and Inductive Bias

Part I has given us everything a course could promise: models with universal expressive power (Chapter 3), losses grounded in maximum likelihood (Chapter 2), an optimizer that scales (Chapter 4), and gradients for anything (Chapter 5). This chapter takes all of it, aims it at real images, and succeeds completely.

Then we will look a little closer, and the wheels will come off. Watching how they come off — in pictures rather than theorems — is the most important lesson of Part I, because the cure is the idea the entire rest of this book is built on.

6.1 Victory

The data is a subset of Fashion-MNIST, a classic benchmark: 28×28 grayscale images of clothing in ten classes. The model and training loop hold no surprises: an MLP from Chapter 3, trained exactly as in Chapter 4:

  1. Prepare the inputs and fixed settings for the example.
  2. Define the reusable helpers: train_mlp and accuracy.
  3. Fit the Chapter 4 recipe on the fixed Fashion-MNIST development split.
  4. Report or visualize the measured result.
import torch
from torch import nn

# [1]
torch.manual_seed(6050)
development = torch.load("../../data/fashion-train.pt")
holdout = torch.load("../../data/fashion-test.pt")
X_dev = development["X"].float().reshape(-1, 784) / 255.0  # (1200, 784)
y_dev, classes = development["y"], development["classes"]
X_test = holdout["X"].float().reshape(-1, 784) / 255.0     # (600, 784)
y_test = holdout["y"]

split = torch.randperm(len(X_dev), generator=torch.Generator().manual_seed(6050))
fit_idx, val_idx = split[:1000], split[1000:]
X_tr, y_tr = X_dev[fit_idx], y_dev[fit_idx]                 # (1000, 784), (1000,)
X_val, y_val = X_dev[val_idx], y_dev[val_idx]               # (200, 784), (200,)

from dlbook.supervised import fit_supervised   # Listing 4.1

# [2]
def train_mlp(
    X: torch.Tensor, y: torch.Tensor, hidden: int = 256,
    epochs: int = 40, seed: int = 6050
) -> nn.Module:
    # The delta from Listing 4.1 is one line: this chapter's model.
    return fit_supervised(
        lambda: nn.Sequential(
            nn.Linear(784, hidden), nn.ReLU(), nn.Linear(hidden, 10)
        ),
        X, y, epochs=epochs, seed=seed,
    )

def accuracy(net: nn.Module, X: torch.Tensor, y: torch.Tensor) -> float:
    net.eval()
    with torch.no_grad():
        return (net(X).argmax(1) == y).float().mean().item()

# [3]
net = train_mlp(X_tr, y_tr)
# [4]
print(f"train accuracy: {accuracy(net, X_tr, y_tr):.1%}")
print(f"validation accuracy: {accuracy(net, X_val, y_val):.1%}")
train accuracy: 95.0%
validation accuracy: 76.0%

The present MLP has no mode-dependent layer, but keep the net.eval() line: it disables training-only behavior such as the dropout mechanism from Chapter 4 before evaluation.

Seventy-six percent on held-out validation images, from one thousand fitting examples and a few seconds of CPU. The split was fixed before any experiment, and the separate test set remains sealed until the end of the chapter. By every metric we have learned to compute, we might be done.

The rest of this chapter is about why we are not.

Before the autopsy, separate three questions that are often collapsed into “the model worked.” Each question needs different evidence.

Table 6.1: Three claims that require three kinds of evidence. Low training error is evidence about one optimization run and, indirectly, the available capacity. It is not a generalization result.
Question Evidence that answers it What that evidence does not settle
Representation: can the model family express a useful rule? A construction, approximation argument, or fitted witness Whether the optimizer will find that rule
Optimization: did this training procedure reach low empirical risk? The loss trace, gradients, restarts, and a final training-set check Behavior on fresh or shifted data
Generalization: does the fitted procedure work on the distribution we care about? Independent evaluation drawn from that declared distribution Performance under an undeclared deployment shift

6.2 Two experiments that should worry you

Experiment 1: move everything two pixels

Take the validation images and shift each one two pixels to the right. Nothing about any garment changes: a coat is a coat, wherever it hangs in the frame. Any human would call the original and shifted batches equivalent for classification. Before you run the cell, commit to a number: the model scores 76% in place — what do you expect at two pixels?

  1. Define the reusable shift_right helper.
  2. Shift validation images while keeping the trained MLP fixed.
# [1]
def shift_right(X: torch.Tensor, px: int) -> torch.Tensor:
    img = X.reshape(-1, 28, 28)
    out = torch.zeros_like(img)
    if px > 0:
        out[:, :, px:] = img[:, :, :-px]
    else:
        out = img.clone()
    return out.reshape(-1, 784)

# [2]
for px in [0, 2]:
    print(f"shift {px}px: validation accuracy "
          f"{accuracy(net, shift_right(X_val, px), y_val):.1%}")
shift 0px: validation accuracy 76.0%
shift 2px: validation accuracy 46.5%

Two pixels, and the accuracy is nearly cut in half. Push further and the collapse completes:

This is the distributional distinction prepared in Chapter 1. If \(T\) shifts an image while preserving its label, then

\[ R_{T_\#P}(\theta) = \E_{(X,Y)\sim P} \left[\ell\bigl(f_\theta(T(X)),Y\bigr)\right], \tag{6.1}\]

where \(T_\#P\) is the distribution obtained by transforming inputs drawn from \(P\). The MLP generalized moderately from its fitting sample to fresh clean examples from \(P\), but it failed the desired robustness contract under \(T_\#P\). There is no contradiction. Clean validation and shift robustness answer different questions. The model did not violate its literal empirical objective; that objective was insufficient for the semantic and deployment contract we later imposed. Appendix E gathers the \(P\), \(T_\#P\), and deployment-distribution notation in one place.

  1. Prepare the inputs and fixed settings for the example.
  2. Plot accuracy vs. shift.
import matplotlib.pyplot as plt

# [1]
shifts = list(range(5))
accs = [accuracy(net, shift_right(X_val, s), y_val) for s in shifts]
plt.figure(figsize=(5.6, 3.3))
# [2]
plt.plot(shifts, accs, "o-", color="#E57200", lw=2, ms=7)
plt.axhline(0.1, ls=":", color="#B8B8A8")
plt.text(0.1, 0.115, "chance (10 classes)", color="#8A8A7A", fontsize=9)
plt.xlabel("shift (pixels right)"); plt.ylabel("validation accuracy")
plt.ylim(0, 0.9); plt.xticks(shifts)
plt.tight_layout(); plt.show()
Validation accuracy falls sharply with each additional pixel of right shift, moving from a strong clean score toward the dotted ten-class chance line by four pixels.
Figure 6.1: The validation cliff. Each pixel of horizontal shift destroys accuracy although the garments themselves never changed. By four pixels the model is approaching guesswork.

Experiment 2: destroy the images entirely

Now the stranger experiment, which exposes a fatal flaw. Choose one fixed random permutation of the 784 pixel positions and apply it to every image, fit and validation alike. The displayed arrays are not recognizable pictures anymore — the ordinary grid adjacency has been destroyed. Retrain the same MLP on this scrambled world — and predict first: does the destroyed geometry cost this model five points, twenty, or nothing?

  1. Prepare the inputs and fixed settings for the example.
  2. Retrain on one fixed permutation of the pixel coordinates.
  3. Report or visualize the measured result.
# [1]
torch.manual_seed(0)
pixel_perm = torch.randperm(784)          # one fixed scrambling for all images

# [2]
net_shuffled = train_mlp(X_tr[:, pixel_perm], y_tr)
# [3]
print(f"original images:  validation accuracy {accuracy(net, X_val, y_val):.1%}")
print(f"shuffled pixels:  validation accuracy "
      f"{accuracy(net_shuffled, X_val[:, pixel_perm], y_val):.1%}")
original images:  validation accuracy 76.0%
shuffled pixels:  validation accuracy 77.0%

The two validation accuracies are close in this one deterministic run. That is not an uncertainty estimate, but it is enough for the structural point: after retraining, an invertible relabeling of pixel coordinates costs this MLP no visible accuracy.

Hold the two results side by side and the diagnosis writes itself:

The MLP hypothesis class and training procedure show no preference for the structure that matters, including pixel adjacency and image geometry, yet the fitted model is hypersensitive to a nuisance that should not matter: where the garment happens to sit. It learned the dataset without needing an image-aware representation.

6.3 The autopsy, in pictures

Why exactly does shifting hurt a model that shuffling cannot? Look at what the model actually learned. Each row of the first layer’s weight matrix is a 784-vector — which means we can reshape it into a 28×28 image and look at it.1 And we know from Chapter 1 what such a row is: a template, scored against the input by dot product.

  1. Select the largest-norm first-layer templates.
# [1]
W1 = net[0].weight.detach()               # (256, 784)
order = W1.norm(dim=1).argsort(descending=True)[:16]
scale = float(W1[order].abs().max())
Sixteen full-frame weight maps fill a two-by-eight grid. Dark red cells mark positive weights, dark blue cells mark negative weights, and pale cells are near zero; broad diffuse patterns span most of each frame.
Figure 6.2: Sixteen first-layer templates, reshaped into images. The labeled scale runs from negative weights at the blue end through pale zero to positive weights at the red end; magnitude grows darker away from zero. Each template is a global, full-frame pattern, and many are diffuse rather than recognizable garment detectors. A template that spans the whole frame changes its score when the input pattern moves.

There is the whole story in one figure. Every template is global — a pattern stretched across the entire frame, tuned to where garments sat in the training images. Shift the input two pixels and every pixel of every template now scores against the wrong part of the garment; hundreds of learned template scores change at once. Conversely, the fixed permutation is merely an invertible relabeling of coordinates. Retraining can permute the first-layer weights in the opposite direction, so the same functions remain available even though adjacency has been destroyed. A model trained on ordinary images is not itself invariant to arbitrary pixel permutations.

Capacity was never the issue. By Chapter 3’s universal approximation theorem, this network can represent a perfectly shift-tolerant classifier. It has no reason to find one: nothing in its architecture, loss, or data ever told it that position is a nuisance and adjacency is meaningful.

6.4 Naming the disease

With the failure in hand, the classical concepts of Part I snap into focus.

Hunting the U-curve

The textbook story (Chapter 1’s cartoon) predicts that as capacity grows, validation error should fall and then rise as the model overfits. We now have everything needed to test the cartoon on real data: a width sweep at two dataset sizes:

  1. Prepare the inputs and fixed settings for the example.
  2. Measure validation error across widths and fitting-set sizes.
# [1]
widths = [2, 4, 8, 16, 64, 256]
# [2]
width_errors = {
    n: [1 - accuracy(train_mlp(X_tr[:n], y_tr[:n], hidden=h, epochs=60),
                       X_val, y_val)
        for h in widths]
    for n in (300, 1000)
}
Both validation-error curves fall steeply as hidden width increases and then flatten. The smaller fitting set turns slightly upward at the widest models, while the larger fitting set does not.
Figure 6.3: Hunting the U-curve: validation error vs. hidden width at two fitting-set sizes. Error falls steeply as capacity grows, then flattens. In this one seeded sweep, n=300 shows a small late upturn while n=1000 does not. The result describes this regime; it does not estimate a scaling law.

Read it honestly. The left side of the cartoon is emphatically real: too little capacity underfits badly. But the dreaded right side barely materializes, with a whisper of an upturn at the small dataset and nothing at the larger one. This is exactly the caveat planted in Chapter 1: the U is a helpful cartoon, not a law of nature. The optimizer, sample size, and data structure can all shape the curve. This one seeded Adam sweep does not identify their separate effects, and it does not establish a \(1/n\) variance law.

Which sharpens our problem instead of solving it: our model does not fail for having too much capacity. It fails for having no knowledge. No point on either curve survives a two-pixel shift.

The curse of dimensionality

Could we fix everything with data alone, just by showing the model garments at every position? Count what that costs. Our images live in 784 dimensions; a small color image (32×32×3) needs 3,072 numbers, a one-megapixel photo three million, an ImageNet input 150,528. And covering a \(d\)-dimensional space densely requires exponentially many examples: if 9 points cover a 1-D interval at some resolution, matching that density takes \(9^2\) points in 2-D and \(9^d\) in \(d\) dimensions. For \(d = 784\), there are not enough atoms in the universe, at any exchange rate.

High dimensionality also breaks the very idea of “similar examples”:

  1. Prepare the inputs and fixed settings for the example.
  2. Measure the nearest-to-farthest distance ratio by dimension.
# [1]
torch.manual_seed(6050)
dims = [2, 10, 100, 784, 5000]
ratios = []
# [2]
for d in dims:
    P = torch.rand(300, d)
    D = torch.cdist(P, P)
    ratios.append(((D + torch.eye(300) * 1e9).min(1).values / D.max(1).values).mean())
The average nearest-to-farthest distance ratio rises toward one as dimension increases on a logarithmic horizontal axis. A vertical guide at dimension 784 meets the curve near 0.89.
Figure 6.4: Distance concentration for 300 points sampled uniformly from a unit cube: the average ratio of nearest-neighbor to farthest-neighbor distance, by ambient dimension. At d=784 the ratio reaches 0.89. This is a controlled geometry example, not a claim that natural images fill the 784-dimensional cube uniformly.

In the uniform cube, brute-force interpolation loses genuinely near neighbors as the ambient dimension grows. Natural images are highly structured and may concentrate near a much lower-dimensional set, so this experiment is not their empirical distance distribution. It shows the price of trying to cover an unconstrained 784-dimensional space and why useful structure must enter through data, representation, or architecture.

6.5 The cure has a name

Return to the word planted on the very first pages of this book (Chapter 1): inductive bias — the nudge that steers a learner toward the functions we want, among the infinitely many that fit. Our MLP’s failure is not a capacity problem or (fixably) a data problem. It is a knowledge problem: we knew things about images that we never told the model.

Say the two principles out loud:

  1. Locality. Nearby pixels are related; a pixel and its neighbor jointly form edges, textures, parts. Distant pixels, far less so. (The shuffle experiment proved our MLP holds no such belief.)
  2. Translation structure. A local detector’s response should move with the feature (equivariance), while a classifier’s final verdict should tolerate or ignore small changes in position (invariance). The shift experiment showed that our fitted MLP did neither reliably.

Data augmentation: put the belief in the examples

The shift cliff suggests another route. If moving a garment a little should not change its class, let \(T\) be a random label-preserving transformation drawn from a declared distribution \(\mathcal{A}\). The population objective becomes

\[ R_{\mathcal A}(\theta) = \E_{(X,Y)\sim P} \E_{T\sim\mathcal{A}} \left[\ell\bigl(f_\theta(T(X)),Y\bigr)\right]. \tag{6.2}\]

This is data augmentation, and its role is larger than making the dataset look bigger. It is a data-side inductive bias: the transformation distribution states which changes we believe should preserve the target. Sampling shifts from \(-3\) to \(+3\) pixels says that location within that window is a nuisance. The empirical augmented objective used in training replaces the outer expectation with the sample average. Drawing one fresh \(T\) when an example enters a batch then gives a Monte Carlo estimate of the inner expectation. Exercise 2 asks you to put that belief into the training batches and measure both what it buys on the shift curve and what it costs on clean images. The corresponding estimator and distribution contracts are summarized in Appendix E.

We can now place three familiar levers side by side:

  1. Architecture-side bias changes which functions are available or easy to learn. Local connectivity and shared convolutional filters are the route Part II will take.
  2. Objective- or training-side bias changes which fitted solution is preferred. Weight penalties, dropout, and early stopping from Chapter 4 live here.
  3. Data-side bias changes the examples the learner is asked to agree on. Augmentation encodes a chosen family of label-preserving transformations.

These routes can work together, but they do not make identical promises. A finite set of augmented shifts encourages consistency; it does not prove exact invariance. An architectural constraint can make a symmetry exact in a layer, but may exclude useful functions. A penalty prefers some solutions without specifying which input changes should preserve a label. And augmentation is only as honest as its transformation: flipping a garment may be harmless, while flipping a digit or a directional sign may change the answer.

For Part II we begin with the architecture-side move: build locality and translation structure into the network itself as constraints. Constrain each template to look at a small local patch instead of the whole frame. Then share that template across every position, so the detection rule itself does not depend on where. Two constraints, and both pathologies are attacked at their root. The parameter count falls too: our MLP has 203,530 parameters, while the LeNet model in Chapter 8 will have 61,706, less than a third as many. Sharing is a massive economy.

TipConstraints are knowledge

A constraint looks like a reduction in power: the constrained model is a strict subset of the MLP. That is exactly the point. By Chapter 1’s bias–variance lens, we are spending bias — assumptions we are confident of — to collapse the search space onto functions that respect the structure of images, slashing variance and sample complexity. Inductive bias is not merely a limitation of a model; done right, it is what you know about the world, cast into architecture, objective, training procedure, or data. And it is a dial, not a dogma: give a weakly-biased model enough data and it can win again — a trade we will meet at the frontier in Chapter 16.

6.6 The pivot

Look once more at the templates figure. The fix is almost visible in it: the templates should not span the frame. They should be small and local — and they should slide, scoring every neighborhood of the image with the same little pattern, so that a sleeve found anywhere is the same sleeve.

A local template, slid across the image, scoring each position by dot product. That machine has a name; it predates deep learning by decades, and Part II opens by building it with our bare hands — then asking this book’s favorite question about it:

What if the template were learnable?

6.7 One sealed test check

The architecture, optimizer, shift size, and reported metrics are now fixed. Only now do we fit the same MLP on all 1,200 development examples and open the 600-example test set once:

  1. Run one final test-set check.
  2. Report or visualize the measured result.
# [1]
net_final = train_mlp(X_dev, y_dev)
# [2]
print(f"clean test accuracy:   {accuracy(net_final, X_test, y_test):.1%}")
print(f"shift-2 test accuracy: "
      f"{accuracy(net_final, shift_right(X_test, 2), y_test):.1%}")
clean test accuracy:   80.8%
shift-2 test accuracy: 42.0%

The sealed check confirms the diagnostic rather than creating it: 80.8% on clean test images becomes 42.0% after the predeclared two-pixel shift.

NoteCheck yourself

Close the book for one minute and retrieve the argument before reading the recap.

  • What two experiments reveal that the MLP learned coordinates rather than image structure?
  • Where can inductive bias enter: architecture, objective, or data? Give one example of each.
  • Why can high clean validation accuracy coexist with collapse after a two-pixel shift?

6.8 Okay, so — the lesson of Part I

  1. Every metric can say yes while the model has learned the wrong thing. Our MLP scored well on clean held-out data while its hypothesis class showed no adjacency preference (fixed shuffle) and its fitted predictions were hypersensitive to a nuisance shift.
  2. The autopsy is visual: global templates, tuned to position, blind to adjacency.
  3. Capacity is not the culprit in this experiment. The U-curve barely appears in one seeded sweep, while the uniform-cube example shows why unconstrained ambient coverage becomes expensive.
  4. The cure is inductive bias: encode knowledge through architecture, objective, training procedure, or data. Part II chooses locality and translation structure as architectural constraints; augmentation is the data-side alternative and companion.

Deeper dive

Networks can memorize random labels. Zhang et al. reported 100% training accuracy on randomly labeled CIFAR-10 and 95.20% on randomly labeled ImageNet under their finite training budget, not 100% on ImageNet. Capacity is abundant enough that fitting noise is easy (Exercise 4 reproduces the phenomenon on our subset). Capacity alone therefore does not explain generalization; the data, optimizer, and architecture must enter the account.

The bias–variance picture, updated. Our flattened U-curve is a mild case of a broader phenomenon: with modern training, test error can even descend again beyond the interpolation point — “double descent.” The cartoon survives as intuition for the underfit regime; the overparameterized regime plays by subtler rules.

Further reading

Sources and further reading

Exercises

  1. (Pencil.) Explain algebraically why the pixel shuffle costs the MLP nothing: if \(\matr{P}\) is a permutation matrix and we train on \(\matr{P}\vect{x}\) instead of \(\vect{x}\), exhibit the weight matrix that makes the two networks identical. Why does no analogous argument work for the shift?
  2. (Code.) Augment the training set with random shifts of ±3 pixels and retrain. Re-plot the cliff. How much robustness did augmentation buy, what did it cost in clean accuracy and compute, and — the real question — what data-side inductive bias did augmentation encode, compared to building invariance into the architecture?
  3. (Code.) The U-curve hunt used widths up to 256. Push to 1024 and 4096 at \(n = 300\), repeating each width across several seeds. Then compare Adam with minibatch SGD from Chapter 4. Which patterns are stable enough to interpret?
  4. (Code.) Random-label audit. (a) Randomly permute the training labels and train a wide MLP for long enough. Report train and test accuracy. (Audit.) (b) Use Table 6.1 to state exactly what successful fitting supports. Explain why one successful run proves neither that the objective is convex, nor that bad local minima are absent, nor that the fitted rule will generalize.
  5. (Pencil.) Estimate the first-layer parameter count of an MLP with 1,000 hidden units on an ImageNet-scale input (224×224×3). At one float32 per parameter, how much memory is that — and how does the number change if each unit instead sees only a 7×7×3 patch?
  6. (Pencil.) Count the labelings of \(N\) ordered points realizable by
    1. positive rays on a line, (b) intervals on a line, and (c) affine linear separators for \(N\) points in general position in two dimensions. Obtain \(N+1\), \(N(N+1)/2+1\), and \(2\sum_{i=0}^{2}\binom{N-1}{i}\), respectively. Compare each with \(2^N\). Do not invoke VC dimension: the point is to see, with arithmetic, how restricting the hypothesis menu changes its growth.

  1. Looking at first-layer weights as images is a long-standing diagnostic; the autopsy below applies it to our own model.↩︎