Sparser, Better, Deeper, Stronger: Improving Static Sparse Training with Exact Orthogonal Initialization
We develop an exact orthogonal initialization technique for static sparse training that enables more robust sparse neural network training.
Jul 1, 2024