-
Loss Masking, Properly Tested: Why the First Experiment Failed and the Second One Worked (Ep:05.06)
A properly designed experiment — more data, a task requiring genuine rule-learning, a real held-out test set — finally shows loss masking’s real benefit clearly, and reveals a precise mechanistic…
-
Loss Masking Explained: Why Fine-Tuning Shouldn’t Train on the Prompt (Ep:05.05)
Instruction fine-tuning data is a (prompt, response) pair, but training loss should only ever be computed on the response tokens. Module 5 of From Zero to Agents derives loss masking…
