Large Language Models

Universal Properties of Activation Sparsity in Modern Large Language Models

We propose a general framework for assessing sparsity robustness in modern LLMs and conduct a systematic study of activation sparsity such models. Our study reveals universal patterns of sparsity in LLMs and provides practical guidelines for model acceleration and design.

Dec 6, 2025

Do LLMs Understand the Safety of Their Inputs? Training-Free Moderation via Latent Prototypes

We develop an efficient approach LLM input safety moderation using latent prototypes and demonstrate that safe and unsafe inputs are separable in the model's latent space.

Feb 22, 2025