From Lines to Networks

We begin with the smallest complete learning system: a model, a loss, and an update rule. At first, the features are fixed, so a linear or logistic model can only recombine what someone chose to measure. A bend between layers makes intermediate representations learnable; stochastic gradient descent moves the parameters, and backpropagation tells each one how it contributed to the error. That flexibility is real, but it carries no knowledge of the world by itself. When a well-fit MLP falls off a cliff after a two-pixel shift, the failure points to the next move: build trusted structure into the architecture.