-
How Attention Replaced Recurrence: The Full Story, From RNNs to a Working GPT
RNNs, LSTMs, attention, transformer blocks, positional encoding, causal masking — six ideas that each fixed a specific, provable failure in the one before it, ending in a complete working language…
-
Building a Tiny GPT From Scratch: Causal Masking and a Complete Working Model (Ep:04.05)
The only architectural difference between an encoder and a GPT-style decoder is one additive mask before softmax. Module 4 of From Zero to Agents derives causal masking, verifies it against…