-
The One-Line Difference Between an Encoder and a GPT-Style Decoder
BERT and GPT are often described as fundamentally different architectures. The actual difference in the attention computation itself is a single additive term. Here’s what that term does, and why…
-
Building a Tiny GPT From Scratch: Causal Masking and a Complete Working Model (Ep:04.05)
The only architectural difference between an encoder and a GPT-style decoder is one additive mask before softmax. Module 4 of From Zero to Agents derives causal masking, verifies it against…
