We challenge the use of calibration metrics in early-exit models and show cases where calibration fails to accurately reflect the network performance. We argue for failure prediction as a more reliable performance proxy that better correlates with efficiency gains in early-exit networks.
Dec 6, 2025
We propose a novel generative mixture-of-GANs approach for accelerating particle detector simulations that maintains high fidelity while achieving significant computational speedups compared to traditional methods.
Aug 30, 2025
We investigates intermediate representations in neural networks during class-incremental learning and propose to leverage them via auxiliary early-exit classifiers. Interestingly, we find out that in continual learning scenarios networks enhanced with such classiers are not only more efficient, but also show improved performance and reduced forgetting across task sequences.
Jul 1, 2025
We propose a method to convert dense transformers to dynamic Mixture-of-Experts models, which leverages natural activation sparsity in the neural networks. Crucially, we propose to enforce activation sparsity during short (continual) training process via additional sparsity regularization, and argue for use of dynamic-k expert routing in MoEfied models. Finally, we show how with efficient implementation our method achieves computational efficiency while maintaining the performance.
Dec 1, 2024
We propose Zero-Time Waste, an early exit network architecture that reduces computational waste via cascading connections between early-exit classifiers and ensembling mechanism. ZTW achieves better efficiency-accuracy trade-offs in pre-trained models and offers a practical architectural solution for deployment of early exit neural networks.
Dec 1, 2023