-
Mini-Batch Gradient Descent Explained: Why Batching Is Just Matrix Multiplication (Ep:03.05)
Batched backpropagation isn’t a new algorithm — it’s the same four backprop equations with vectors replaced by matrices, where matrix multiplication sums over the batch automatically. Module 3 of From…
-
Backpropagation From Scratch: Deriving the Four Equations, Verified Against PyTorch (Ep:03.04)
Backpropagation is four equations, applied layer by layer. Module 3 of From Zero to Agents derives all four from the chain rule, implements them in raw NumPy with no autograd,…
-
Why Weight Initialization Matters: Xavier and He Init, Derived From Scratch (Ep:03.03)
The wrong starting weights can make a deep network’s activations vanish to zero or explode to millions before training even begins. Module 3 of From Zero to Agents derives Xavier…
-
The Vanishing Gradient Problem Explained: Why ReLU Replaced Sigmoid (Ep:03.02)
Sigmoid’s derivative maxes out at 0.25 and shrinks fast — multiply that through 15 layers and gradients vanish to nearly zero before reaching early layers. Module 3 of From Zero…
-
How Multi-Layer Neural Networks Solve XOR (and Why Nonlinearity Is Non-Negotiable) (Ep:03.01)
Stacking two perceptrons solves the problem one perceptron provably can’t. Module 3 of From Zero to Agents hand-builds a working XOR network, proves why linear layers alone can never work…
-
The Perceptron and the XOR Problem: Why Single Neurons Aren’t Enough (Ep:03.00)
In 1969, Minsky and Papert proved a single perceptron can never learn XOR. Module 3 of From Zero to Agents derives that proof from scratch, trains a real perceptron in…
-
Why Your Training Loss Exploded to NaN: The Math of Learning Rate, Not Guesswork
A loss curve that spikes to NaN mid-training isn’t random instability — it’s a provable consequence of crossing an exact convergence threshold. Here’s the derivation, and what it means for…
-
The Math Behind Every LLM, in Five Steps: Vectors to Cross-Entropy Loss
One continuous derivation connects vectors, matrices, gradients, and probability into the exact training objective behind every modern language model. Here’s the full arc, with the reasoning that forces each step.
-
Softmax and Cross-Entropy Explained: The Loss Function Behind Every LLM (Ep:02.04)
Softmax turns raw scores into a probability distribution; cross-entropy measures how wrong that distribution is. Module 2 of From Zero to Agents derives both from probability theory, proves the famous…
-
Why Learning Rate Matters: Gradient Descent Convergence, Proven With a Closed-Form Equation (Ep:02.03)
A learning rate that’s too large doesn’t just train slower — it can make loss diverge to infinity, provably. Module 2 of From Zero to Agents derives the exact convergence…
-
Derivatives and Gradients — The Mechanism Behind “Learning” (Ep:02.02)
Every neural network “learns” by following a gradient downhill. Module 2 of From Zero to Agents builds derivatives, partial derivatives, and the chain rule from first principles, verifies them against…
-
Matrices — Organized, Simultaneous Dot Products (Ep:02.01)
A matrix is just organized, simultaneous dot products. Module 2 of From Zero to Agents builds matrix multiplication from first principles and uses it to fully decode the real attention…
