Deep Learning: Making It Learnable
What to Learn, What to Build In, and What to Reuse
A first-principles deep learning textbook built around one compounding question: what if we made this learnable? From linear regression to the pretrained era.
Preface
A first-principles course in Python and PyTorch.
Much of modern deep learning can be understood as familiar machinery, made learnable, placed inside the right structure, and run on hardware that makes matrix multiplication absurdly fast.
What to learn. What to build in. What to reuse.
Why now
For many readers trained in statistics, the foundational lesson was to keep the model small relative to the data. Too many adjustable parameters would fit noise as readily as signal, and high-dimensional estimation demanded defensive care. Deep learning steps squarely into the regime classical intuition warned against: overparameterized models with millions or billions of parameters, optimized across vast, non-convex loss landscapes. Yet on unseen data, these models routinely generalize.
Hardware specialized for matrix multiplication makes this regime accessible. Linear algebra organizes the computation into operations that accelerators execute with high efficiency, while backpropagation computes exact gradients for all those parameters in a single reverse sweep.
The deeper surprise, however, is that additional capacity can improve out-of-sample prediction. In many measured regimes, increasing parameters, data, and compute in a balanced ratio produces continuing power-law reductions in held-out loss. Generalization need not collapse when a network interpolates its training sample. Architecture, data distribution, and training dynamics still govern success, but parameter count alone no longer marks where useful learning must stop.
We will encounter this surprise in Chapter 6 and fit its empirical scaling laws in Chapter 19. A complete theoretical accounting of why high-dimensional learners generalize remains an active research frontier. Our task here is to understand the mechanics: how to design the architecture, how to wire the computation, and how to audit what it has learned.
As my math PhD advisor used to remind me: sometimes you just have to follow your nose. When a model or an objective feels overwhelming, you do not wait for an epiphany. You pick up a pencil, work through the derivations line by line, check the shapes, and trust the mathematical mechanics to carry you forward. The intuition is earned through the work, not before it. Understanding the computation should make its success more interesting, not make that success seem inevitable.
What the mechanics explain, and what they do not
Building, wiring, and running these models takes familiar mathematics: the chain rule, matrix products, vector–Jacobian products, and probability for the losses. Explaining why they work so well at scale is a different matter. Networks with far more parameters than training examples can fit random labels, yet on real data they generalize; the double descent met in Chapter 6 is one face of this. Gradient descent on a nonconvex loss in millions or billions of dimensions routinely finds solutions that generalize. Over wide ranges of scale, loss falls roughly as a power law in parameters, data, and compute, a regularity that Chapter 19 fits and then qualifies. Accounting for these facts draws on high-dimensional probability, random matrix theory, dynamical systems, and statistical physics, and much of it is still active research.
This book neither pretends those questions are settled nor waits for them. It is a builder’s and auditor’s manual: fluency in the mechanics and the inductive biases lets you read that theory, and the claims made in its name, with clear eyes.
A continuation, not a fresh start
You own more of this subject than you think. If you know linear regression, empirical risk minimization, gradient descent, and why squared error is the maximum-likelihood loss for Gaussian noise, you are not starting over. Each architecture in this book keeps the model before it and makes one more piece learnable, shared, or reused.
Each rung keeps the model below it and adds one move, shown in orange. The grey tags name the Part of the book where each move is made.
Take a linear model and let it learn its own nonlinear features: the core of a multilayer perceptron. Take fixed convolutional filters and learn them from data: the core of a convolutional network. Take a one-shot summary and learn how to update it step by step: recurrence. Take similarity-weighted averaging and learn how to compare and mix: the core of attention. Learn a representation once and adapt it many times: the pretrained era.
The underlying mathematical move, in its simplest instance:
\[ \underbrace{f_{\vect{w}}(x) = \vect{w}^\top \phi(x)}_{\text{linear model on fixed features}} \qquad\longrightarrow\qquad \underbrace{f_{\vect{w},\theta}(x) = \vect{w}^\top \phi_\theta(x)}_{\text{linear readout on learned nonlinear features}}. \]
What we build in, through architecture, objectives, data, or training, is inductive bias: regularities the model should not have to rediscover from scratch. Three questions organize everything that follows:
\[ \begin{aligned} \textbf{What is learned?} &\quad \text{features, filters, comparisons, representations} \\ \textbf{What is built in?} &\quad \text{locality, sharing, order, invariance, visibility} \\ \textbf{What is reused?} &\quad \text{pretrained representations and models} \end{aligned} \]
With these questions in hand, architectures stop looking like a zoo of named models and start reading as a short sequence of design moves. When an idea feels half-familiar, it usually is. Follow your nose backward: the cross-references first, then Appendix A for linear algebra and the SVD, Appendix B for tensors, and Appendix E for statistical learning.
Open the hood
This book spends little time persuading you that deep learning matters. It spends its time on how learning works. Treat a network as an engine to assemble, tune, and repair:
Forward, data flows left to right; backward, sensitivities flow right to left. Every layer holds learnable parameters (orange), and the loss compares the prediction (green) with the target (purple).
- Forward, left to right. Data flows through linear maps, one batched matrix multiplication (GEMM) per layer, each followed by a nonlinear bend.
- Backward, right to left. Reverse-mode differentiation carries the loss’s sensitivity back to every parameter, one vector–Jacobian product per layer.
- The design goal. As much representational power as possible from as few parameters as possible, obtained by building the data’s known structure into the architecture.
When training stalls, we do not guess. We check shapes and strides, gradient norms layer by layer, what each position is allowed to see, and whether the numbers are well conditioned.
Who this book is for
This book is for builders and auditors: engineers, students, and researchers who want to understand deep learning well enough to implement it from scratch, diagnose why it fails, and evaluate its claims with clear eyes.
You do not need previous deep learning experience, but working fluency with multivariable calculus, linear algebra, basic probability, and standard Python will let you focus on the architecture rather than the background math. The opening chapters revisit the necessary optimization and loss mechanics, while the appendices provide self-contained reference contracts: linear algebra and the SVD (Appendix A), tensors and PyTorch layout (Appendix B), numerical precision and the Roofline model (Appendix C), notation (Appendix D), and statistical learning contracts (Appendix E).
While our executable code uses PyTorch, the principles are strictly framework-independent. Tensor shapes, vector–Jacobian products, memory hierarchies, and broadcasting rules apply identically whether you write PyTorch, JAX, or TensorFlow. (And should you prefer another dialect, modern AI coding assistants are remarkably adept at translating clean syntax.)
How to read this book
Use the same loop throughout: read → predict → run → audit. Before executing an example, write down the shape, direction, or trend you expect. Then compare the result with your prediction. When they disagree, reduce the problem to one batch or one example and find where your mental model of the computation broke down.
Five questions organize every audit:
- Function class. What family of functions does the code implement, and what structure is built in (locality, weight sharing, order, causality)?
- Objective. What does the loss reward, and which conditional noise model justifies it?
- Estimator. What does each batch actually estimate, i.e., an unbiased per-example mean, a biased plug-in of an aggregate, or an objective defined by the batch itself? Chapter 4 separates the three cases.
- Visibility. What information can the model see? Check padding, causal masks, and leakage across data splits.
- Claim. What does the experiment actually establish, and which tempting conclusions remain unproven?
Apply these questions whether you wrote the code by hand, borrowed it from a repository, or generated it with an AI assistant. Assistants now write plausible PyTorch in seconds, so syntax is no longer the bottleneck → auditing is. An assistant that writes code you cannot audit costs you the very competence you came here to build.
The code is plain Python and PyTorch and runs on an ordinary laptop CPU because the data are small. If you are taking the course, the lecture videos and course-site self-checks complement this loop.
For a first pass, read the opening problem, study the figures, run the single cell that tests the central claim, and answer the Check yourself prompts without looking back. Then return for the derivations, code, and exercises. The Trap and tip boxes help diagnose common bugs.
When you encounter an unfamiliar architecture in a newly published paper, you should be able to identify what it changes, trace its computation, and design a decisive test of its claims. That is the fluency this book asks you to build.
Why this book
Goodfellow, Bengio, and Courville’s Deep Learning is the encyclopedic reference; Prince’s Understanding Deep Learning emphasizes visual explanation; Dive into Deep Learning ranges broadly across frameworks and tools; Bach’s Learning Theory from First Principles develops generalization theory from the ground up. This book is narrower on purpose. It follows one question through every architecture, what if we made this learnable?, and prints only code that ran, so reading, predicting, running, and auditing form one loop.
This book is complete on its own terms. It tells the architecture story as one arc, from linear regression to the pretrained era, and teaches exactly the training mechanics its experiments need: loss and stochastic gradient descent, backpropagation, and numerical precision. It also leaves you ready for the research literature. With these mechanics in hand, current papers on test-time learning, state-space sequence models, diffusion, and preference optimization become readable.
Readers who want the theory of training will find it in Shakeri (2026), Deep Learning: Making It Trainable, a separate, self-contained treatment of initialization, optimization dynamics, numerical contracts, and the spectra of high-dimensional data that shares this book’s notation.
Make the right parts learnable. Build in the structure you trust. Reuse what transfers.
This is the free, open book for DS 6050 Deep Learning at UVA’s School of Data Science. The course arc is published, and the book continues to receive corrections and teaching improvements. The current stable edition is v1.4 (September 28, 2026); its tagged source preserves a fixed version. The live book is a rolling post-v1.4 manuscript. Since v1.4 the interludes are numbered chapters, so chapter numbers from 7 on differ from v1.3’s; the revision notes map the old numbers to the new. The HTML edition on this website is the canonical text. Executable figures and numerical results are produced by code in the book’s source: experiment code sits in expandable panels, revealed step by step or all at once, while the source of pure concept diagrams stays in the repository rather than on the page. Every printed snippet is the code that ran; nothing on the page is retyped, and the repository’s checks re-execute the book’s notebooks. Source, provenance, and licenses remain visible in the repository.
Suggested citation: Shakeri, Heman. 2026. Deep Learning: Making It Learnable. Version 1.4. https://shakeri-lab.github.io/dl-book/.
The route at a glance
| Part | What was fixed | What becomes learnable | The failure that opens the next part |
|---|---|---|---|
| I · From Lines to Networks | the input features | intermediate representations | global templates span the whole frame, so accuracy falls off a cliff after a small shift |
| II · Vision: Learning the Filters | hand-designed local filters | detector weights | a one-shot fixed-width code supplies no rule for updating it as a sequence arrives |
| III · Sequences: Learning the Summary | a one-shot summary | the carried state update | one fixed-size encoder–decoder bridge must carry every source fact |
| IV · Attention: Learning the Similarity | the similarity rule | the comparison rule | full fine-tuning copies or updates the whole pretrained backbone per task |
| V · The Pretrained Era: Learning What to Reuse | the reuse rule or sampler | adaptation, a learned judge, and generation | objectives, data, and evaluation remain choices no architecture makes |
Course route and dependencies
The route is cumulative. Modules sometimes revisit a chapter because the course uses the same idea first as a mechanism and later as evidence. The appendices are just-in-time support, not a separate prerequisite block: A is linear algebra and the SVD, B is tensors in practice, C is numerical precision and hardware efficiency, D is notation, and E is statistical learning contracts.
| Course module | Primary book route | Supporting appendices |
|---|---|---|
| 1 · foundations and MLPs | 1 → 2 → 3 → 4 | A · B · D · E |
| 2 · backpropagation | 5 | B · D |
| 3 · optimization and ablation | 4 → 6 → experiment interlude | B · D · E |
| 4 · convolutional networks | 6 → 8 → 9 | B · D · E |
| 5 · modern CNNs and transfer | 10 | B · C · D |
| 6 · PCA and autoencoders | PCA → autoencoder interlude | A · B · D |
| 7 · recurrent sequences | 12 → 13 | B · D |
| 8 · attention | 14 → 15 | A · B · D |
| 9 · Transformers | 16 → test-time-regression interlude | B · C · D |
| 10 · pretrained models, ViTs, scaling | 18 → 19 | B · C · D |
| 11 · adaptation and alignment | 20 → 21 | B · C · D · E |
| 12 · generative and multimodal models | 22 → 23 | A · B · C · D · E |
Plan a reading session
Time depends more on whether you stop to derive and run than on page count. As a planning range, allow roughly 45–90 minutes for a careful first read of most chapters, with the experiment-heavy later chapters often taking the longer end or a second sitting. Once the environment and data are ready, reproducing a few key cells commonly adds 20–45 minutes; rerunning every training cell from scratch can take substantially longer and varies by machine. Exercises are best treated as their own work session.
If you have one short sitting, read the opening problem, study the figures, run one cell that tests the main claim, and answer any closing Check yourself prompts without looking back. Return for the derivations and exercises when you can explain what failed, what changed, and what evidence supports the repair.
Exercises, hints, and solutions
The book supports both enrolled and self-paced readers. For-credit assignment rules on the course site and in the syllabus take precedence. To protect those assignments, the public book does not publish full worked solutions to overlapping graded work. Public Check yourself prompts are retrieval practice, not answer banks; the chapter recap and course-site self-check feedback provide the first layer of hints.
For self-paced work, use these acceptance criteria:
- A conceptual answer names the mechanism, predicts a direction or limiting case, and states one condition under which the claim could fail.
- A shape derivation labels every axis before the operation and checks the resulting shape against the code.
- A code exercise runs from a fresh session, fixes its seed, prints or asserts the important shapes, and reproduces the requested qualitative relationship. Small machine-dependent numerical differences are not failures unless the exercise sets a tolerance.
- An experiment states the question before the run, separates tuning from final evaluation, keeps the comparison recipe fixed, reports seed variation when requested, and ends with a limitation rather than a universal claim.
When stuck, inspect the nearest Trap: or tip callout, write the expected shape or trend before running anything, and reduce the code to one batch or one example. Enrolled students should use the course’s approved help channels for assignment-specific hints; complete graded solutions stay there rather than in the public text.
Acknowledgments
To the students of DS 6050 at the University of Virginia School of Data Science, whose questions and debugging pauses shaped every explanation here.
Support this open book
This book is free to read at $0, and no contribution unlocks additional content. Reading it, sharing it, or reporting a correction is already meaningful support. If it has been useful and you would like to help sustain ongoing corrections, new figures, and open releases, you may make an optional contribution here:
Text and figures licensed CC BY-NC-SA 4.0; all code MIT. Source on GitHub. The graduate companion is Deep Learning: Making It Trainable.
A release is a fixed place to point a syllabus, assignment, or citation. The live book may continue to receive corrections, but a tagged release keeps its source fixed. When the exact text matters, name the version.
Rolling build · October 9, 2026 · The print edition’s additions
New material from the print edition now appears here too. The Transformer chapter (Chapter 16) writes the attention score as one learned matrix and draws one row of attention as geometry in a new figure, so its later figures renumber. It also explains why the original Transformer needed warmup, what attention alone does to a stack of tokens, and why the residual additions keep tokens distinct. New exercises treat the temperature softmax (Chapter 2), forbidding tokens one step at a time (Chapter 13) and the regression function (Appendix E — Statistical Learning Contracts). The attention interlude (Chapter 17) shows why softmax’s kernel has no finite feature map, the alignment chapter (Chapter 21) adds programmatic checks and reward hacking, and the generative chapter (Chapter 22) says that a sampled result depends on the sampler as well as the weights. No printed result changed.
Rolling build · October 8, 2026 · The print edition’s later revisions
The print edition’s revisions of October 6 to 8 now appear here too. The factor \(\sigma'(z)\) in backpropagation is called the activation derivative, and “gate” names only the LSTM and GRU gates. Seed numbers stay in the code, not the prose. Chapters 6 to 13 take the author’s rewritten passages on capacity and pixel geometry, experiments, convolution, channels and padding, the autoencoder’s loss, recurrence, and sequence-to-sequence training and search, and the sequences part opens with the author’s new introduction. Retitled sections keep their old web addresses. No printed result changed.
Rolling build · October 5, 2026 · The print edition’s revisions
The author’s revisions to the Springer print edition now appear here too, chapter by chapter: plainer setups, claims scoped to the runs that support them, and the corrections to claims, equations and headings that the print review found. Chapter 7 is now titled “Interlude: Learning by Experiment” and Chapter 21 “Instruction Tuning and Preference Learning”; retitled sections keep their old web addresses. Figures 7.1, 8.1, 12.3, 17.1 and 17.2 were redrawn from their unchanged data. No printed result changed.
Rolling build · September 30, 2026 · Parts II and IV prose
Chapters 8, 9, 10, 14, 15, 16, 18 and 19 read more plainly. A label before a colon is now a clause, sentences that only announced the next one are gone, and the word ledger has left their prose. A work named in the running text now links where it is first named, and Chapter 10’s sources add the GoogLeNet paper. Chapter 14 walks its opening example through the table that maps kernel regression onto sequence memory. In Section 16.1, “The position debt” is now “Self-attention does not see order”; links to the old heading still work. Figure 9.3 now names the receptive field and adds the 6×6 field after the first pool. Figure 10.3 draws the identity stream as the straight line and the learned branch below it, with \(H(x)=x+F(x)\) before the final ReLU, as the code computes it, and the DenseNet aside now follows the residual block it compares against. No printed result changed.
Rolling build · September 30, 2026 · Chapter 22
Chapter 22’s headings now name the three model families, and diffusion is one section with the forward and reverse processes as its two parts, so Sections 22.5 to 22.7 are now 22.4 to 22.6. The VAE pages say that the diagonal Gaussian is a choice and that reparameterization reaches any location–scale family. The GAN pages derive the optimal discriminator point by point and show the step to the Jensen–Shannon form. Old links to the renamed sections still work, and no printed result changed.
Rolling build · September 30, 2026 · Figures in code panels
A figure or results table produced by a Plan → Code cell now shows while that cell’s code is closed. Twenty figures had stayed hidden until the reader opened the code, among them Figures 8.2 and 12.2. The source, its printed results, and any table still open only from a plan step, Show all code, or Reveal results. The replay beside Figure A.2, which had been hidden with the code, now shows too. No text or printed result changed.
Rolling build · September 29, 2026 · Figure 8.2
Figure 8.2 now prints the nine weights of each kernel above the images that kernel produces, so identity, blur, sharpen, and the two Sobel kernels read as numbers as well as effects. The images and every printed result are unchanged.
Rolling build · September 29, 2026 · Figure 10.1
Figure 10.1 is redrawn with one row for each block design the chapter builds. Each row composes a block, repeats it, and adds a head, and every box matches a layer in the chapter’s code. The residual row’s identity route now runs forward into the sum instead of looping back. No code or printed result changed.
Rolling build · September 28, 2026 · Appendix A
Appendix A’s least-squares example now names its solver, a plain QR factorization. With PyTorch’s default choice, the last digits of the example’s residual could change from one run to the next. The residual now reads about \(6.7\times10^{-16}\); the solution and every other printed result are unchanged.
Rolling build · September 28, 2026 · Chapters 13 and 16
Chapter 13 reads more directly, and its Figure 13.1 again fits the page. Chapter 16 gains two schematics: Figure 16.1 follows one query through a sequence that queries itself, and Figure 16.3 draws sinusoidal position as a bank of clocks. The later figures of Chapter 16 renumber; printed results are unchanged.
Rolling build · September 28, 2026 · The reference machine
Every printed number now comes from one machine at one thread count. The chapters’ setup cells pin the CPU thread count, and the autoencoder interlude was re-executed on that machine: two of its tables move in the fourth decimal place or later, and two of its figures follow, while every comparison its text draws still holds.
v1.4 · September 28, 2026 · Interludes numbered as chapters
The three interludes are now numbered chapters: Learning by Experiment is Chapter 7, Making PCA Learnable is Chapter 11, and Attention as Test-Time Regression is Chapter 17. The chapters after each interlude renumber, so the former Chapter 7 is now Chapter 8 and the former Chapter 20 is now Chapter 23; every chapter reference in the text follows. Web addresses and anchors are unchanged. The notes below use the chapter numbers of their time.
| v1.3 | v1.4 | Chapter |
|---|---|---|
| 1 to 6 | 1 to 6 | unchanged |
| Interlude | 7 | Interlude: Who Trains the Trainer? Learning by Experiment |
| 7 | 8 | Filters and Convolution (Fixed Kernels) |
| 8 | 9 | CNNs: Making the Filters Learnable |
| 9 | 10 | Modern CNNs and Transfer Learning |
| Interlude | 11 | Interlude: Autoencoders: Making PCA Learnable |
| 10 | 12 | Sequences and Recurrence |
| 11 | 13 | Encoder–Decoder, Teacher Forcing, Beam Search |
| 12 | 14 | Kernel Regression: Attention Before It Was Learnable |
| 13 | 15 | Attention: Making the Kernel Learnable |
| 14 | 16 | Self-Attention and the Transformer |
| Interlude | 17 | Interlude: Attention as Test-Time Regression |
| 15 | 18 | The BERT Moment: Pretraining as the New Regime |
| 16 | 19 | Vision Transformers and Scaling Laws |
| 17 | 20 | Adapting Pretrained Models: Prompting, PEFT, Quantization |
| 18 | 21 | Alignment and RL Fine-Tuning |
| 19 | 22 | Generative Models: From Codes to Samples |
| 20 | 23 | Multimodal Learning: One Space, Two Views |
Version 1.4 also lets the book stand on its own. Headings and exercise labels that read “non-examinable” now read “optional”; Deep Learning: Making It Trainable is cited as a separate work; Chapter 10’s transfer rule states its 224-pixel landmark case as a prediction rather than a measured result; and the numbers drawn inside figures, printed in code, and written into the downloadable notebooks follow the new chapter numbers. It also gathers the rolling changes listed below since v1.3, among them the Part III revision of Chapters 12 and 13 and their renumbered exercises.
Rolling build · September 26, 2026 · HTML edition only
The website now serves the HTML edition only: the PDF download page and both PDF files are gone, and neither the sidebar nor this Preface offers a PDF. Chapters, figures, notebooks, and printed outputs are unchanged.
Rolling build · September 26, 2026 · Chapter 1 setup cell
Chapter 1’s first code cell no longer displays the random-number generator it seeds. That display showed a memory address that changed on every run, so it was the one output that differed between the HTML and PDF editions and between builds. All other printed output is unchanged.
Rolling build · September 26, 2026 · Chapter 1 footnotes
Chapter 1’s three footnotes, on bias augmentation, on why Gaussian noise is a reasonable model for residuals, and on the radial shrink behind weight decay, had reached only the PDF editions. They now appear in the HTML edition as well, with the same text, and both PDF editions still carry them. Printed experimental outputs and the archived v1.3 release are unchanged.
Rolling build · September 25, 2026 · Preface, Chapters 1 to 7, and references
The Preface now says why this moment in deep learning rewards study, separates the mechanics this book teaches from the open theory of why they work at scale, draws the learnability ladder and the forward and backward passes as two new figures, and states the five audit questions explicitly. Chapter 1 shows the gradient behind the normal equations and reads the projection from both sides; Chapter 2 names the classification head, adds an input-space view of the sigmoid’s boundary, and names the KL divergence; Chapter 3 reframes XOR’s history around credit assignment and shows how depth folds space; Chapter 4 opens the two zones of SGD on their mechanism and derives momentum from a ball with friction. Chapter 5 now develops backpropagation as a recursion on layer sensitivities, with redrawn figures, a new figure of four autograd rules in PyTorch syntax, the batch form of the weight gradient, and the cost of the backward pass in arithmetic and in memory; its BatchNorm exercise moved to Chapter 9, where BatchNorm is introduced. Chapter 6 completes its two experiments into a two-by-two protocol, separates cropping from moving in the shift experiment, shows why a dense layer cannot see pixel adjacency, and cites the theorem behind confident predictions far from the training data. The experimentation interlude’s opening figure now draws both levels of learning: the training run inside the experiment cycle. Chapter 7 now states directly why it builds convolution with fixed kernels first and what changes when the kernel is learned. Across the book, every code panel’s plan now names what its steps do. A full reference audit corrected two dead links, one table number, three title–link mismatches, three attributions, one year, and two wording slips; no cited work was found to be fabricated. A punctuation pass removed nearly every em dash. Printed experimental outputs are unchanged, except that Chapter 14’s training printout again matches the numbers its text reports; the archived v1.3 release is unchanged.
Rolling build · September 13, 2026 · Search and mechanism replays
Chapter 11 now includes a small probability tree showing how the best next token can miss the best complete sequence. Both PDF editions carry the example; the optional HTML replay contrasts greedy and beam search on the same fixed tree. It joins the book’s lightweight mechanism replays, with supplied histories kept distinct from candidate continuations. Existing experimental outputs and the archived v1.3 release remain unchanged.
Rolling build · September 2, 2026 · Phase F
The code path now continues beyond reading. Twenty-six chapter, interlude, and foundational-appendix pages provide a downloadable notebook and an Open in Colab route. Each notebook is generated from the canonical manuscript, preserves its learner-visible Plan → Code sequence, omits hidden plotting harnesses, and begins with a commit-pinned bootstrap for exact runtime versions and SHA-256-verified artifacts.
Before publication, six clean CI shards execute every compact notebook beside a full Quarto-derived reference from the same manuscript. Their learner-visible stdout must agree byte for byte on that runner; a reviewed exception list then checks it against the frozen HTML evidence, with exact comparison as the default. Only the unexecuted source notebooks that pass both checks are published. The Preface, Epilogue, and two reference appendices say that no executable notebook is available; Part pages carry no notice. This pipeline changes neither manuscript content nor the tagged v1.3 PDFs.
Rolling build · September 2, 2026 · Phase E
Four remaining web-reading details change in one audited pass. Collapsed Plan → Code source remains searchable: finding a token opens the panel and selects the plan step that owns its bracketed code region. MathJax typesets inline and unnumbered mathematics near the viewport while keeping numbered citation targets eager; the first content image remains eager while later images load and decode lazily, and the download page prefers a 230 KB WebP cover with the original PNG as its fallback.
These are HTML-only presentation and delivery changes. Equation numbers, equation links, source alternatives, and the existing Show-all path remain intact; the tagged v1.3 PDFs and their page counts are unchanged.
v1.3 · September 2, 2026
Version 1.3 opens each Part with a short transition that names the fixed object, the learnable move, the structure built in, and the failure that motivates what follows. Part III becomes Sequences: Learning the Summary, and Part V becomes The Pretrained Era: Learning What to Reuse; the sidebar, route table, HTML pages, PDF openers, and outline agree.
This release also fixes the post-v1.2.1 front door and publication work: the canonical HTML states the book’s independent scope, exposes a deterministic edition stamp, and keeps both derived PDFs free behind a cover-led download page. The two 0.85-inch PDF profiles are fixed at 548 print pages and 519 continuous-screen pages. All 390 outline destinations land on their rendered headings, all 133 frozen stdout blocks remain unchanged, and the structural, accessibility, text-layer, glyph, and geometry audits pass in both formats.
Rolling build · September 2, 2026
Every manuscript page now carries a direct link to its source, without adding a second global code-visibility control. Thirty non-Part reading pages also carry a compact HTML-only chapter-tools strip: the most specific public DS 6050 lecture resources available, plus a reserved notebook slot for the next publication phase. Twenty-seven pages have a specific video, slide deck, or course segment; the other three cover the whole book and use the course playlist. The five Part pages carry no strip. These web affordances begin the rolling manuscript after v1.3; the two released v1.3 PDFs remain fixed.
Rolling build · September 1, 2026
The canonical HTML repairs Chapter 14’s Sources heading and gives every figure an accessible description. Every rendered page now carries a canonical URL and a source-derived rolling/stable edition stamp; book pages also begin with a keyboard-first skip link. The publication audit rejects raw Markdown leaks, unresolved cross-references, and missing image alternatives before deployment.
Rolling build · August 20, 2026
The canonical HTML now keeps ancillary material closed until a reader opens it: About this edition, Revision notes, and the left-sidebar chapter groups, while the active chapter’s own Part remains available for orientation. The sidebar also opens a cover-led download page directly under the book title. That page keeps both PDF editions free and links its suggested $20 contribution directly to the external support page, without inserting a redundant local amount picker. The support heading gains one small decorative coffee mark. Narrative tips, traps, and retrieval checks remain open.
Rolling build · August 19, 2026
Both derived PDFs now open with the author-supplied cover while retaining their searchable title pages and title verso. The shared Preface also closes with one optional support invitation: the complete book remains free at $0, no contribution unlocks additional content, and readers may help sustain corrections, new figures, and open releases through one stable link.
Rolling build · August 8, 2026
The canonical HTML now carries book-level description, favicon, social-card, and machine-readable citation metadata; MathJax is pinned to the exact renderer used by this build. Interlude tables complete the EX. namespace, Plan → Code panels add an explicit Show all code path, the link palette clears WCAG AA, selected prose-like outputs wrap on narrow screens, and the two softest executed figures are regenerated at high density. A book-specific 404 page keeps stale links inside the reading path.
Rolling build · August 6, 2026
The live post-v1.2.1 book adds Appendix E as an optional statistical-contract reference, replaces Chapter 1’s bias–variance cartoon with a seeded show-then-name experiment, separates representation, optimization, and generalization evidence in Chapter 6, and bounds Chapter 18’s alignment contract within a broader sociotechnical assessment. The stable v1.2.1 release remains unchanged.
v1.2.1 · August 2, 2026
Version 1.2.1 links this book and its graduate companion volume in both directions. Ten chapter anchors are now protected as public interfaces in source and rendered-HTML audits; five bounded forward pointers separate mechanics owned here from the diagnostic extensions in Deep Learning: Making It Trainable. The two books also point to each other from their colophons.
v1.2 · July 29, 2026
Version 1.2 is the comprehensive audit and convention-parity release. It gives all three interludes independent figure namespaces and retrieval checks, completes the book-wide Plan → Code and exercise-taxonomy contracts, strengthens estimator and evidence discipline, adds the attention-as-test-time-regression interlude, and closes the temperature thread from kernel bandwidth through training-time scale and post-hoc calibration. The epilogue now has its own E. figure namespace and cites primary sources. Its attached 552-page PDF is the stable page-reference target; the canonical HTML and derived PDF passed the complete manuscript, frozen-output, glyph, and cross-reference audits at release.
v1.1 · July 24, 2026
Version 1.1 is the readable-code and estimator-discipline release. It establishes the equation/kernel/harness visibility rule, transcludes canonical executable listings, adds the learned-feature and forget-gate diagnostics, strengthens batch-estimator discipline, and adds a weekly from-scratch execution audit. Its attached 502-page PDF is the v1.1 page-reference target.
v1.0 · July 16, 2026
Version 1.0 is the first stable, citable edition. It closes the July 2026 structural and research-frontier revision with the complete twenty-chapter course arc, both bridge interludes, the epilogue, and four just-in-time appendices. It also pins the Chapter 1–9 corpus used by the Chapter 10/14 language-model rematch, completes the figure-description audit, and applies one presentation rule consistently: experiments expose their code, while pure concept diagrams keep executable drawing source in the repository without printing pages of coordinates.
Pre-v1.0 builds moved page numbers while the middle of the book was restructured and the final chapters were added. Migrate old page references to the PDF attached to the v1.0 release. Later notes will describe any page-reference changes relative to this baseline. Machine-readable citation metadata lives in CITATION.cff.