-
How LSTMs Fix the Vanishing Gradient Problem: The Gating Math, Derived and Measured (Ep:04.01)
An LSTM’s cell state has a gradient path that doesn’t pass through a matrix multiply or a squashing nonlinearity at every timestep — that’s the actual mechanism that fixes vanishing…
-
RNNs and the Vanishing Gradient Problem: Why Attention Had to Be Invented (Ep:04.00)
An RNN reuses the same weight matrix at every timestep, which means the vanishing gradient problem from Module 3 comes back — across time instead of depth, and often much…
-
The Vanishing Gradient Problem Explained: Why ReLU Replaced Sigmoid (Ep:03.02)
Sigmoid’s derivative maxes out at 0.25 and shrinks fast — multiply that through 15 layers and gradients vanish to nearly zero before reaching early layers. Module 3 of From Zero…
