-
Where to Apply LoRA in a Transformer: Allocating a Fixed Rank Budget Optimally (Ep:05.02)
For a fixed total parameter budget, splitting LoRA rank equally across weight matrices is usually wrong — some matrices need far more adaptation capacity than others. Module 5 of From…
-
The One-Line Difference Between an Encoder and a GPT-Style Decoder
BERT and GPT are often described as fundamentally different architectures. The actual difference in the attention computation itself is a single additive term. Here’s what that term does, and why…
-
The Transformer Block Explained: Multi-Head Attention, Residuals, and Layer Norm (Ep:04.03)
Residual connections fix vanishing gradients across depth the same way LSTM gating fixed them across time — both give the gradient a near-identity path. Module 4 of From Zero to…
