-
How LSTMs Fix the Vanishing Gradient Problem: The Gating Math, Derived and Measured (Ep:04.01)
An LSTM’s cell state has a gradient path that doesn’t pass through a matrix multiply or a squashing nonlinearity at every timestep — that’s the actual mechanism that fixes vanishing…
-
RNNs and the Vanishing Gradient Problem: Why Attention Had to Be Invented (Ep:04.00)
An RNN reuses the same weight matrix at every timestep, which means the vanishing gradient problem from Module 3 comes back — across time instead of depth, and often much…
-
Why Your Classifier Trains So Slowly at First (and MSE Might Be Why)
A classification model that’s confidently wrong early in training can get an almost nonexistent correction signal if you’re using the wrong loss function. Here’s the exact mechanism, measured, and why…
-
How a Neural Network Actually Learns: From a Single Neuron to Backpropagation
A single artificial neuron can’t solve XOR. Here’s the full, provable chain of fixes — nonlinearity, activation functions, initialization, backpropagation, batching, and the right loss function — that turns that…
-
Why Cross-Entropy Beats MSE for Classification: The Gradient Math, Proven (Ep:03.06)
A confidently wrong sigmoid+MSE network gets a weaker correction signal exactly when it needs a stronger one. Module 3 of From Zero to Agents derives why softmax+cross-entropy avoids this pathology,…
-
Mini-Batch Gradient Descent Explained: Why Batching Is Just Matrix Multiplication (Ep:03.05)
Batched backpropagation isn’t a new algorithm — it’s the same four backprop equations with vectors replaced by matrices, where matrix multiplication sums over the batch automatically. Module 3 of From…
-
Backpropagation From Scratch: Deriving the Four Equations, Verified Against PyTorch (Ep:03.04)
Backpropagation is four equations, applied layer by layer. Module 3 of From Zero to Agents derives all four from the chain rule, implements them in raw NumPy with no autograd,…
-
Why Weight Initialization Matters: Xavier and He Init, Derived From Scratch (Ep:03.03)
The wrong starting weights can make a deep network’s activations vanish to zero or explode to millions before training even begins. Module 3 of From Zero to Agents derives Xavier…
-
The Vanishing Gradient Problem Explained: Why ReLU Replaced Sigmoid (Ep:03.02)
Sigmoid’s derivative maxes out at 0.25 and shrinks fast — multiply that through 15 layers and gradients vanish to nearly zero before reaching early layers. Module 3 of From Zero…
-
How Multi-Layer Neural Networks Solve XOR (and Why Nonlinearity Is Non-Negotiable) (Ep:03.01)
Stacking two perceptrons solves the problem one perceptron provably can’t. Module 3 of From Zero to Agents hand-builds a working XOR network, proves why linear layers alone can never work…
-
The Perceptron and the XOR Problem: Why Single Neurons Aren’t Enough (Ep:03.00)
In 1969, Minsky and Papert proved a single perceptron can never learn XOR. Module 3 of From Zero to Agents derives that proof from scratch, trains a real perceptron in…
-
Why Your Training Loss Exploded to NaN: The Math of Learning Rate, Not Guesswork
A loss curve that spikes to NaN mid-training isn’t random instability — it’s a provable consequence of crossing an exact convergence threshold. Here’s the derivation, and what it means for…
