Why Al Models Forget & How MIT Fixed It With Knowledge Retention

MIT introduces Self-Distillation Fine-Tuning to reduce catastrophic forgetting; it uses student-teacher demonstrations and needs 2.5x compute.