LoRA Explained: How Low-Rank Adaptation Actually Works
Changing rank without updating alpha silently halves your effective learning rate → Attention-only target modules is a 2021 default — MLP layers matter more → High rank on small data doesn’t improve accuracy; it causes overfitting → LoRA can degrade safety alignment even on clean datasets