Catastrophic Forgetting
Catastrophic forgetting is the tendency of a neural network to abruptly lose previously learned knowledge when it is trained on new data or tasks, because the weight updates that help it learn the new task overwrite the parameters that…
Definition
Catastrophic forgetting is the tendency of a neural network to abruptly lose previously learned knowledge when it is trained on new data or tasks, because the weight updates that help it learn the new task overwrite the parameters that encoded the old one.
Overview
Catastrophic forgetting was first documented in connectionist neural network research in the late 1980s, and it remains one of the central obstacles to building AI systems that learn continuously rather than being trained once and frozen. The core issue is structural: a standard network represents everything it knows as a single shared set of weights, so gradient descent and backpropagation on a new dataset shift those same weights toward the new objective, with no built-in mechanism to protect the parameters that mattered for earlier tasks. The problem shows up most clearly in sequential or 'continual learning' settings — a model trained on task A, then task B, then task C, tends to perform well on C but poorly on A and B, even though it once handled them fine. It is distinct from ordinary overfitting: the model isn't failing to generalize within one dataset, it is actively losing a capability it previously had. This matters a great deal for fine-tuning workflows, where teams adapt a pretrained LLM to a narrow domain and can find that broad, general capabilities degrade in the process — sometimes called 'alignment tax' or capability regression in the LLM context. Mitigations fall into a few broad families: regularization approaches that penalize large changes to weights deemed important for old tasks (such as elastic weight consolidation), replay-based approaches that mix a sample of old training data back in during new training, and architectural approaches that allocate new capacity (extra parameters or modules) for new tasks while freezing the old ones — an idea related to techniques like LoRA, which trains small adapter layers instead of the full model precisely to limit how much of the original network is disturbed. Courses that cover model adaptation in depth, such as PyTorch Deep Learning, typically address this trade-off when discussing fine-tuning strategy.
Key Concepts
- Occurs when new training overwrites weights that encoded previously learned tasks
- Most visible in sequential or continual learning setups, not single-dataset training
- Distinct from overfitting — the model loses a capability it once had, not just generalization
- A known risk when fine-tuning large pretrained models on narrow domains
- Mitigated by regularization methods that protect important prior weights
- Mitigated by replay strategies that mix old data back into new training
- Mitigated by architectural approaches that add new capacity instead of overwriting shared weights