Skip to main content
What you'll learn
Builds on Modules 1–10
≈7 h · 2 h video · 3 h reading · 2 h coding
  • Choose zero-shot, few-shot, or chain-of-thought prompting for a concrete task
  • Explain how RAG grounds generation and when retrieval is preferable to changing weights
  • Compare full fine-tuning with LoRA/QLoRA in trainable parameters and memory use
  • Distinguish instruction tuning, preference learning, and policy optimization as alignment objectives
  • Use the course decision framework to choose prompting, RAG, PEFT, or quantization
11.1 – ICL, Prompt Engineering, and RAG

Slide preview

View Slides
11.2 – Parameter-Efficient Fine-Tuning (PEFT)

Slide preview

View Slides
11.3 – Quantization & QLoRA

Slide preview

View Slides
📚 Resources & Lecture Code

Course book · primary reading

🧪 Gemma Fine-Tuning

Gemma model card: ai.google.dev/gemma/docs/core/model_card_3

Full Model Fine-Tune (Hugging Face Transformers, Colab)Fine-Tune with QLoRA (Hugging Face, Guide)Fine-Tuning with Gemma Library (Colab)
🧠 Test your understanding

Try each question before revealing the answer — these mirror the ideas the module quiz checks.

Q1.What is the central idea of In-Context Learning (ICL) in large language models?
  • Updating model weights during inference
  • Learning tasks from examples provided in the prompt without parameter updates
  • Distilling a smaller model from a larger one
  • Retrieving documents from an external database
Show answer ▾

Answer: Learning tasks from examples provided in the prompt without parameter updates

ICL is learning by conditioning on demonstrations in the prompt with frozen parameters. The model leverages attention to infer the task from exemplars, allowing the same model to perform new tasks without modifying weights.

Q2.The defining feature of Retrieval-Augmented Generation (RAG) is:
  • Fine-tuning on retrieved passages
  • Plugging an external retriever to ground generation on found documents
  • Using a smaller tokenizer
  • Quantizing the model to 4-bit
Show answer ▾

Answer: Plugging an external retriever to ground generation on found documents

RAG separates knowledge storage from parametric memory by dynamically injecting retrieved documents into the context during generation, improving freshness and reducing hallucinations without changing model weights.

Q3.In LoRA, a weight matrix W∈Rdout×dinW \in \mathbb{R}^{d_{\text{out}} \times d_{\text{in}}} is adapted using which parameterization?
  • W′=αWW' = \alpha W
  • W′=W+ΔWW' = W + \Delta W with ΔW=BA\Delta W = BA, where A∈Rr×dinA \in \mathbb{R}^{r \times d_{\text{in}}} and B∈Rdout×rB \in \mathbb{R}^{d_{\text{out}} \times r}
  • W′=W⊤W' = W^\top
  • W′=W−λ∇WW' = W - \lambda \nabla W
Show answer ▾

Answer: W′=W+ΔWW' = W + \Delta W with ΔW=BA\Delta W = BA, where A∈Rr×dinA \in \mathbb{R}^{r \times d_{\text{in}}} and B∈Rdout×rB \in \mathbb{R}^{d_{\text{out}} \times r}

LoRA trains low-rank factors AA and BB while keeping the base weight WW frozen. At inference, the adapted output is W′x=Wx+B(Ax)W'x = Wx + B(Ax), where rank r≪min⁡(din,dout)r \ll \min(d_{\text{in}}, d_{\text{out}}) controls adapter capacity and reduces trainable parameters to approximately r(din+dout)r(d_{\text{in}} + d_{\text{out}}).

Q4.QLoRA fine-tunes large language models by:
  • Training all weights in FP32
  • Storing the base weights in 4-bit and training LoRA adapters in 16-bit
  • Using only prompt engineering
  • Distilling to a tiny student model
Show answer ▾

Answer: Storing the base weights in 4-bit and training LoRA adapters in 16-bit

QLoRA combines quantization of base weights (e.g., to NF4 4-bit) with LoRA adapters trained in higher precision. This achieves near full fine-tuning quality on consumer GPUs by dramatically reducing memory requirements while maintaining task-specific expressivity.

Q5.In a decision framework, choose RAG over PEFT when:
  • The model must remember static domain style forever
  • You need strict data governance and freshness without changing weights
  • You cannot build an index
  • You have abundant fine-tuning labels
Show answer ▾

Answer: You need strict data governance and freshness without changing weights

RAG is ideal when knowledge changes frequently or is sensitive/private, since it retrieves up-to-date information at inference without modifying weights. PEFT is better for persistent behavior, style, or domain shifts that must remain constant across calls.

Module 10
Module 12