-
How Attention Replaced Recurrence: The Full Story, From RNNs to a Working GPT
RNNs, LSTMs, attention, transformer blocks, positional encoding, causal masking — six ideas that each fixed a specific, provable failure in the one before it, ending in a complete working language…
-
How LSTMs Fix the Vanishing Gradient Problem: The Gating Math, Derived and Measured (Ep:04.01)
An LSTM’s cell state has a gradient path that doesn’t pass through a matrix multiply or a squashing nonlinearity at every timestep — that’s the actual mechanism that fixes vanishing…
