-
Perplexity Can Lie: Why Which Tokens You Evaluate On Changes the Answer (Ep:05.07)
Full-sequence perplexity makes an unmasked-trained model look better than a masked-trained one — the opposite of the truth. Module 5 of From Zero to Agents derives perplexity from cross-entropy, then…
-
Loss Masking Explained: Why Fine-Tuning Shouldn’t Train on the Prompt (Ep:05.05)
Instruction fine-tuning data is a (prompt, response) pair, but training loss should only ever be computed on the response tokens. Module 5 of From Zero to Agents derives loss masking…
-
The Math Behind Every LLM, in Five Steps: Vectors to Cross-Entropy Loss
One continuous derivation connects vectors, matrices, gradients, and probability into the exact training objective behind every modern language model. Here’s the full arc, with the reasoning that forces each step.
