Adapt Your Teacher: Improving Knowledge Distillation for Exemplar-Free Continual Learning
We examine knowledge distillation in exemplar-free continual learning and find out that allowing the adaptation of teacher network during the learning process through batch normalization updates improves knowledge transfer across several continual learning methods.
Jan 1, 2024