Asked about Running a fine-tune properly
What is catastrophic forgetting?
An answer that works
Fine-tuning on a narrow task degrading the model's general abilities. LoRA reduces it by leaving the original weights untouched, and mixing general examples into training helps further. Test general capability as well as the target task.
Then they push further
How would you detect it, given that the training loss looks fine?
The training loss is computed on the fine-tuning data, where nothing is going wrong, so it cannot show this. You need a separate held-out set of the general behaviour you want to keep, scored the same way on every run, from the first run rather than added after somebody notices in production. A short fixed set of general instructions is enough.
What do you do about it, and what does each fix cost?
Two levers. Lowering the learning rate preserves more of the old behaviour and learns the new task correspondingly less well, because forgetting and learning are the same process. Replay — mixing examples of the old behaviour into the training set, typically a fifth to a half of it — is the better lever, because it buys back most of the old capability at a small cost on the new task. LoRA puts a ceiling on the damage, since a rank-8 or rank-16 update has nowhere near the capacity to overwrite general behaviour, and it gives you a rollback by simply not serving the adapter.
Reading an answer is not being able to give one. The section that teaches this is Running a fine-tune properly, 6 hours of Fine-tuning.
Learn this properlyAlso asked about Fine-tuning