When Pruning Meets Interpretability: Preserving Sparse Autoencoder Robustness in LLMs

• URL: http://arxiv.org/abs/2608.25941v1
• Date: Wed, 26 Aug 2026 15:57:42 GMT
• Title: When Pruning Meets Interpretability: Preserving Sparse Autoencoder Robustness in LLMs
• Abstract summary: Sparse autoencoders (SAEs) are widely used to interpret the internal representations of large language models.<n>We show that for a fixed SAE, its impact is governed by perturbation energy.<n>We propose a layer-wise sparsity allocation strategy, achieving lower perplexity under the same average pruning sparsity.
• Score: 15.418870425179875
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sparse autoencoders (SAEs) are widely used to interpret the internal representations of large language models (LLMs), yet their reliability under post-hoc model compression remains poorly understood. We present a systematic study of how pruning affects SAE behavior and theoretically show that, for a fixed SAE, its impact is governed by perturbation energy, a covariance-weighted norm. This perspective exposes a key limitation of magnitude pruning: by ignoring activation geometry, it distorts the learned representation space and degrades SAE functionality. Activation-aware methods such as Wanda and SparseGPT, in contrast, implicitly control perturbation energy and are therefore substantially more robust at preserving SAE behavior. We further reveal a consistent structural vulnerability across all pruning methods: middle layers are significantly more sensitive to pruning than early or late layers. Guided by this insight, we propose a layer-wise sparsity allocation strategy, achieving lower perplexity under the same average pruning sparsity. Experiments across four model architectures validate our theoretical findings. Code is publicly available at https://github.com/osu-srml/sae-robustness-under-pruning/tree/main.

Related papers

• From Sparse Features to Trustworthy Proxies: Certifying SAE-Based Interpretability [53.681678236115836]
  We study when can an SAE-based explanation be treated as a faithful view of an underlying frozen LM.<n>Our framework derives an upper bound on the base model's expected risk using four measurable quantities.<n>We show that the bound becomes non-vacuous on GPT-2 Small, Gemma-2B, and Llama-3-8B at practical sample sizes.
  arXiv  Detail & Related papers  (2026-06-16T18:28:23Z)
• Rethinking the Role of Tensor Decompositions in Post-Training LLM Compression [68.1291440835878]
  decompositions have emerged as a promising direction, offering compact parameterizations well suited to Transformer weight structures.<n>Existing studies evaluate these methods in narrow settings, leaving unclear whether tensorization is effective at large-scale deployment.<n>We systematically evaluate tensor compression across dense and MoE, establishing performance trade-offs grounded in both empirical analysis and theoretical analysis.
  arXiv  Detail & Related papers  (2026-06-02T10:45:21Z)
• Towards Understanding the Robustness of Sparse Autoencoders [13.16745936025085]
  We present a study of integrating pretrained SAEs into transformer residual streams at inference time.<n>SAE-augmented models achieve up to a 5x reduction in jailbreak success rate relative to the undefended baseline.
  arXiv  Detail & Related papers  (2026-04-20T19:00:09Z)
• Improving Robustness In Sparse Autoencoders via Masked Regularization [21.265992727881514]
  We propose a masking-based regularization that randomly replaces tokens during training to disrupt co-occurrence patterns.<n>This improves robustness across SAE architectures and sparsity levels reducing absorption, enhancing probing performance, and narrowing the OOD gap.
  arXiv  Detail & Related papers  (2026-04-07T21:56:23Z)
• When Does Sparsity Mitigate the Curse of Depth in LLMs [53.137717161619484]
  We show that sparsity acts as a regulator of variance propagation and thereby improves depth utilization.<n>Our results reveal sparsity as a key yet previously overlooked mechanism for effective depth scaling in large language models.
  arXiv  Detail & Related papers  (2026-03-16T15:04:16Z)
• Sparse Semantic Dimension as a Generalization Certificate for LLMs [53.681678236115836]
  We introduce the Sparse Semantic Dimension (SSD), a complexity measure derived from the active feature vocabulary of a Sparse Autoencoder (SAE) trained on the model's layers.<n>We validate this framework on GPT-2 Small and Gemma-2B, demonstrating that our bound provides non-vacuous certificates at realistic sample sizes.
  arXiv  Detail & Related papers  (2026-02-11T21:45:18Z)
• Analysis of Variational Sparse Autoencoders [1.675385127117872]
  We investigate whether incorporating variational methods into SAE architectures can improve feature organization and interpretability.<n>We introduce the Variational Sparse Autoencoder (vSAE), which replaces deterministic ReLU gating with sampling from learned Gaussian posteriors.<n>Our findings suggest that naive application of variational methods to SAEs does not improve feature organization or interpretability.
  arXiv  Detail & Related papers  (2025-09-26T23:09:56Z)
• Normalized Attention Guidance: Universal Negative Guidance for Diffusion Models [57.20761595019967]
  We present Normalized Attention Guidance (NAG), an efficient, training-free mechanism that applies extrapolation in attention space with L1-based normalization and refinement.<n>NAG restores effective negative guidance where CFG collapses while maintaining fidelity.<n>NAG generalizes across architectures (UNet, DiT), sampling regimes (few-step, multi-step), and modalities (image, video)
  arXiv  Detail & Related papers  (2025-05-27T13:30:46Z)
• Effective Layer Pruning Through Similarity Metric Perspective [0.0]
  Deep neural networks have been the predominant paradigm in machine learning for solving cognitive tasks. Pruning structures from these models is a straightforward approach to reducing network complexity. Layer pruning often hurts the network predictive ability (i.e., accuracy) at high compression rates. This work introduces an effective layer-pruning strategy that meets all underlying properties pursued by pruning methods.
  arXiv  Detail & Related papers  (2024-05-27T11:54:51Z)
• Towards Practical Control of Singular Values of Convolutional Layers [65.25070864775793]
  Convolutional neural networks (CNNs) are easy to train, but their essential properties, such as generalization error and adversarial robustness, are hard to control. Recent research demonstrated that singular values of convolutional layers significantly affect such elusive properties. We offer a principled approach to alleviating constraints of the prior art at the expense of an insignificant reduction in layer expressivity.
  arXiv  Detail & Related papers  (2022-11-24T19:09:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.