Catastrophic Forgetting

Catastrophic forgetting is the tendency of a neural network to lose previously learned abilities when it's trained on new data. Fine-tune a capable base model narrowly on your support tickets and it may get great at your domain while silently getting worse at general reasoning, formatting, or the safety behavior it once had — the new gradients overwrite the weights that encoded the old skills. For SaaS builders this is the hidden risk of fine-tuning: you optimize for one metric and regress on others you weren't measuring. Mitigations include mixing general data back into the fine-tuning set, using lightweight adapters like LoRA that leave the base weights frozen, keeping learning rates low, and — often the simplest answer — avoiding fine-tuning altogether in favor of retrieval and good prompting. If you do fine-tune, always evaluate on a broad held-out suite, not just your target task, so you catch regressions before your users do. A model that aces your domain but forgot how to follow formatting instructions is a net loss.