-
Where to Apply LoRA in a Transformer: Allocating a Fixed Rank Budget Optimally (Ep:05.02)
For a fixed total parameter budget, splitting LoRA rank equally across weight matrices is usually wrong — some matrices need far more adaptation capacity than others. Module 5 of From…