-
How LSTMs Fix the Vanishing Gradient Problem: The Gating Math, Derived and Measured (Ep:04.01)
An LSTM’s cell state has a gradient path that doesn’t pass through a matrix multiply or a squashing nonlinearity at every timestep — that’s the actual mechanism that fixes vanishing…
-
RNNs and the Vanishing Gradient Problem: Why Attention Had to Be Invented (Ep:04.00)
An RNN reuses the same weight matrix at every timestep, which means the vanishing gradient problem from Module 3 comes back — across time instead of depth, and often much…
-
Why Cross-Entropy Beats MSE for Classification: The Gradient Math, Proven (Ep:03.06)
A confidently wrong sigmoid+MSE network gets a weaker correction signal exactly when it needs a stronger one. Module 3 of From Zero to Agents derives why softmax+cross-entropy avoids this pathology,…
-
Mini-Batch Gradient Descent Explained: Why Batching Is Just Matrix Multiplication (Ep:03.05)
Batched backpropagation isn’t a new algorithm — it’s the same four backprop equations with vectors replaced by matrices, where matrix multiplication sums over the batch automatically. Module 3 of From…
-
Backpropagation From Scratch: Deriving the Four Equations, Verified Against PyTorch (Ep:03.04)
Backpropagation is four equations, applied layer by layer. Module 3 of From Zero to Agents derives all four from the chain rule, implements them in raw NumPy with no autograd,…
-
Why Weight Initialization Matters: Xavier and He Init, Derived From Scratch (Ep:03.03)
The wrong starting weights can make a deep network’s activations vanish to zero or explode to millions before training even begins. Module 3 of From Zero to Agents derives Xavier…
-
The Vanishing Gradient Problem Explained: Why ReLU Replaced Sigmoid (Ep:03.02)
Sigmoid’s derivative maxes out at 0.25 and shrinks fast — multiply that through 15 layers and gradients vanish to nearly zero before reaching early layers. Module 3 of From Zero…
-
How Multi-Layer Neural Networks Solve XOR (and Why Nonlinearity Is Non-Negotiable) (Ep:03.01)
Stacking two perceptrons solves the problem one perceptron provably can’t. Module 3 of From Zero to Agents hand-builds a working XOR network, proves why linear layers alone can never work…
-
The Perceptron and the XOR Problem: Why Single Neurons Aren’t Enough (Ep:03.00)
In 1969, Minsky and Papert proved a single perceptron can never learn XOR. Module 3 of From Zero to Agents derives that proof from scratch, trains a real perceptron in…
-
Softmax and Cross-Entropy Explained: The Loss Function Behind Every LLM (Ep:02.04)
Softmax turns raw scores into a probability distribution; cross-entropy measures how wrong that distribution is. Module 2 of From Zero to Agents derives both from probability theory, proves the famous…
-
Why Learning Rate Matters: Gradient Descent Convergence, Proven With a Closed-Form Equation (Ep:02.03)
A learning rate that’s too large doesn’t just train slower — it can make loss diverge to infinity, provably. Module 2 of From Zero to Agents derives the exact convergence…
-
Derivatives and Gradients — The Mechanism Behind “Learning” (Ep:02.02)
Every neural network “learns” by following a gradient downhill. Module 2 of From Zero to Agents builds derivatives, partial derivatives, and the chain rule from first principles, verifies them against…
