-
The One-Line Difference Between an Encoder and a GPT-Style Decoder
BERT and GPT are often described as fundamentally different architectures. The actual difference in the attention computation itself is a single additive term. Here’s what that term does, and why…
-
Building a Tiny GPT From Scratch: Causal Masking and a Complete Working Model (Ep:04.05)
The only architectural difference between an encoder and a GPT-style decoder is one additive mask before softmax. Module 4 of From Zero to Agents derives causal masking, verifies it against…
-
Positional Encoding Explained: How Transformers Learn Word Order (Ep:04.04)
Attention is mathematically blind to sequence order — swap two tokens and every output is just the same values, permuted. Module 4 of From Zero to Agents derives sinusoidal positional…
-
The Transformer Block Explained: Multi-Head Attention, Residuals, and Layer Norm (Ep:04.03)
Residual connections fix vanishing gradients across depth the same way LSTM gating fixed them across time — both give the gradient a near-identity path. Module 4 of From Zero to…
-
Learned Q, K, V Explained: What Makes Attention Actually Trainable (Ep:04.02)
Attention only becomes powerful once the query, key, and value projections are learned rather than reused from static embeddings. Module 4 of From Zero to Agents trains a real attention…
-
Why Your Classifier Trains So Slowly at First (and MSE Might Be Why)
A classification model that’s confidently wrong early in training can get an almost nonexistent correction signal if you’re using the wrong loss function. Here’s the exact mechanism, measured, and why…
-
Mini-Batch Gradient Descent Explained: Why Batching Is Just Matrix Multiplication (Ep:03.05)
Batched backpropagation isn’t a new algorithm — it’s the same four backprop equations with vectors replaced by matrices, where matrix multiplication sums over the batch automatically. Module 3 of From…
-
Backpropagation From Scratch: Deriving the Four Equations, Verified Against PyTorch (Ep:03.04)
Backpropagation is four equations, applied layer by layer. Module 3 of From Zero to Agents derives all four from the chain rule, implements them in raw NumPy with no autograd,…
-
Why Weight Initialization Matters: Xavier and He Init, Derived From Scratch (Ep:03.03)
The wrong starting weights can make a deep network’s activations vanish to zero or explode to millions before training even begins. Module 3 of From Zero to Agents derives Xavier…
-
The Vanishing Gradient Problem Explained: Why ReLU Replaced Sigmoid (Ep:03.02)
Sigmoid’s derivative maxes out at 0.25 and shrinks fast — multiply that through 15 layers and gradients vanish to nearly zero before reaching early layers. Module 3 of From Zero…
-
Why Your Training Loss Exploded to NaN: The Math of Learning Rate, Not Guesswork
A loss curve that spikes to NaN mid-training isn’t random instability — it’s a provable consequence of crossing an exact convergence threshold. Here’s the derivation, and what it means for…
-
From Zero to Agents: A Foundational AI/ML Course Built From First Principles (Episode 00.00)
Most “learn AI” content today teaches you to assemble — wire an LLM API into a framework, call it an agent, ship it. That’s a real and useful skill. It…
