Skip to main content

Module 9 – Transformer

What you'll learn
Builds on Modules 1–8
≈6 h · 2 h video · 2 h reading · 2 h coding
  • Implement a transformer block and train a tiny Transformer
  • Compare positional encodings; pre- vs post-norm; analyze patterns
  • Use warmup and dropout; document stability/perf. impacts
Lecture 9 – Transformer
Lecture 9.2 – Transformer (Part 2)
📚 Resources & Lecture Code

Required reading · course book

Research lens · read after the course book

These papers use the regression lens as architecture-design language; begin with the named sections after completing the book derivation.

Suggested reading:

Open Colab Notebook
Module 8
Module 10