Zero Time Waste in Pre-trained Early Exit Neural Networks

Early exit neural networks enable adaptive inference by allowing samples to exit at different network depths based on prediction confidence. However, existing approaches often waste computational resources during training and inference. We propose a novel training methodology that eliminates this waste, achieving superior efficiency-accuracy trade-offs in pre-trained models while maintaining the flexibility of adaptive computation.

In Neural Networks