Fundamental Limits of Neural Network Sparsification: Evidence from Catastrophic Interpretability Collapse

• URL: http://arxiv.org/abs/2603.18056v1
• Date: Wed, 18 Mar 2026 00:16:38 GMT
• Title: Fundamental Limits of Neural Network Sparsification: Evidence from Catastrophic Interpretability Collapse
• Abstract summary: This work investigates feature survival under severe capacity constraints in Variational Autoencoder--Sparse Autoencoder architectures.<n>We introduce an adaptive sparsity scheduling framework that progressively reduces active neurons from 500 to 50 over 50 training epochs.<n>Our key finding reveals a pervasive paradox: while global representation quality remains stable, local feature interpretability collapses systematically.
• Score: 7.167095059974211
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Extreme neural network sparsification (90% activation reduction) presents a critical challenge for mechanistic interpretability: understanding whether interpretable features survive aggressive compression. This work investigates feature survival under severe capacity constraints in hybrid Variational Autoencoder--Sparse Autoencoder (VAE-SAE) architectures. We introduce an adaptive sparsity scheduling framework that progressively reduces active neurons from 500 to 50 over 50 training epochs, and provide empirical evidence for fundamental limits of the sparsification-interpretability relationship. Testing across two benchmark datasets -- dSprites and Shapes3D -- with both Top-k and L1 sparsification methods, our key finding reveals a pervasive paradox: while global representation quality (measured by Mutual Information Gap) remains stable, local feature interpretability collapses systematically. Under Top-k sparsification, dead neuron rates reach $34.4\pm0.9\%$ on dSprites and $62.7\pm1.3\%$ on Shapes3D at k=50. L1 regularization -- a fundamentally different "soft constraint" paradigm -- produces equal or worse collapse: $41.7\pm4.4\%$ on dSprites and $90.6\pm0.5\%$ on Shapes3D. Extended training for 100 additional epochs fails to recover dead neurons, and the collapse pattern is robust across all tested threshold definitions. Critically, the collapse scales with dataset complexity: Shapes3D (RGB, 6 factors) shows $1.8\times$ more dead neurons than dSprites (grayscale, 5 factors) under Top-k and $2.2\times$ under L1. These findings establish that interpretability collapse under sparsification is intrinsic to the compression process rather than an artifact of any particular algorithm, training duration, or threshold choice.

Related papers

• A Dominant Diffuse Phase in the Sparse Autoencoder Phase Diagram [0.0]
  Sparse autoencoders (SAEs) are increasingly used to recover interpretable features from neural-network activations.<n>The MAIS-O43 open problem proposes a controlled experiment to characterize when recovery of a true synthetic dictionary gives way to feature merging.
  arXiv  Detail & Related papers  (2026-09-09T15:13:42Z)
• SANE: State Anomaly Neutralization for Stable Extreme-Context Delta-Rule Models [53.86918766240095]
  Delta-Rule recurrent models maintain a fixed-size state, enabling $O(1)$ inference memory but potentially becoming unstable under extreme-context extrapolation.<n>We propose textbfState Anomaly Neutralization (SANE), which applies adaptive $tanh$ compression at chunk boundaries while preserving the intra-chunk parallel structure.
  arXiv  Detail & Related papers  (2026-08-23T10:41:07Z)
• From Training to Deployment: Post-Hoc Causal Feature Identification via Sensitivity Ratios [10.912482254600478]
  Given a model that is already trained, which features does it rely on causally versus spuriously?<n>We introduce the textbfNormalised Sensitivity Ratio(NSR), a post-hoc, model-agnostic diagnostic for this question.
  arXiv  Detail & Related papers  (2026-07-28T10:26:09Z)
• Safety, Security, and Cognitive Risks in Neuro-Symbolic AI [0.0]
  Neuro-symbolic AI (NeSy) pairs neural perception with symbolic reasoning.<n>NeSy attacks five layers: neural perception, symbolic knowledge bases, reasoning engines, agentic orchestration, and data stores.
  arXiv  Detail & Related papers  (2026-06-15T19:11:55Z)
• Feature Starvation as Geometric Instability in Sparse Autoencoders [0.764671395172401]
  We introduce adaptive elastic net SAEs, a fully differentiable architecture grounded in classical sparse regression.<n>AEN-SAEs combine an $ell$ structural term that enforces strong convexity and Lipschitz stability with adaptive $ell$ reweighting.<n>We show that AEN-SAEs yield a Lipschitzcontinuous sparse coding map and recover the global feature support under mild assumptions.
  arXiv  Detail & Related papers  (2026-05-06T18:11:14Z)
• ChronoSpike: An Adaptive Spiking Graph Neural Network for Dynamic Graphs [10.05203937088835]
  Spiking neural networks offer event-driven efficiency but remain limited by sequential propagation, binary information loss, and local aggregation that misses global context.<n>We propose ChronoSpike, an adaptive spiking graph neural network that integrates learnable LIF neurons with per-channel membrane dynamics, multi-head attentive spatial aggregation on continuous features, and a lightweight Transformer temporal encoder.<n>On three large-scale benchmarks, ChronoSpike outperforms twelve state-of-the-art baselines by $2.0%$ Macro-F1 and $2.4%$ Micro-F1 while achieving $3-10times$ faster training than recurrent methods with a
  arXiv  Detail & Related papers  (2026-02-01T09:50:21Z)
• Hierarchical Federated Graph Attention Networks for Scalable and Resilient UAV Collision Avoidance [0.5505634045241287]
  Real-time performance, adversarial resiliency, and privacy preservation are the most important metrics that need to be balanced to practice collision avoidance.<n>We have proposed an adaptive differential privacy mechanism, wherein the noise level $(in [0.1, 1.0])$ is dynamically reduced based on an evaluation of the measured real-time threat.<n>This architecture provides a scalable scenario of 500 UAVs with a collision rate of $ 2.0%$ and the Byzantine fault tolerance of $f n/3$.
  arXiv  Detail & Related papers  (2025-11-05T12:01:00Z)
• Beyond Softmax: A Natural Parameterization for Categorical Random Variables [61.709831225296305]
  We introduce the $textitcatnat$ function, a function composed of a sequence of hierarchical binary splits.<n>A rich set of experiments show that the proposed function improves the learning efficiency and yields models characterized by consistently higher test performance.
  arXiv  Detail & Related papers  (2025-09-29T12:55:50Z)
• SoftSignSGD(S3): An Enhanced Optimizer for Practical DNN Training and Loss Spikes Minimization Beyond Adam [28.46753383125989]
  Adam has proven successful in training deep neural networks, but the mechanisms underlying its successes and limitations remain underexplored.<n>In this study, we demonstrate that the effectiveness of Adam stems largely from its similarity to SignSGD in robustly fluctuating fluctuations.<n>We propose $emphThird, S3 incorporates an equivalent Nesterov's accelerated gradient(NAG) module, accelerating convergence without memory overhead.
  arXiv  Detail & Related papers  (2025-07-09T00:47:37Z)
• Global Convergence and Rich Feature Learning in $L$-Layer Infinite-Width Neural Networks under $μ$P Parametrization [66.03821840425539]
  In this paper, we investigate the training dynamics of $L$-layer neural networks using the tensor gradient program (SGD) framework.<n>We show that SGD enables these networks to learn linearly independent features that substantially deviate from their initial values.<n>This rich feature space captures relevant data information and ensures that any convergent point of the training process is a global minimum.
  arXiv  Detail & Related papers  (2025-03-12T17:33:13Z)
• Towards understanding neural collapse in supervised contrastive learning with the information bottleneck method [26.874007846077884]
  Neural collapse describes the geometry of activation in the final layer of a deep neural network when it is trained beyond performance plateaus. We demonstrate that neural collapse leads to good generalization specifically when it approaches an optimal IB solution of the classification problem.
  arXiv  Detail & Related papers  (2023-05-19T18:41:17Z)
• Improved techniques for deterministic l2 robustness [63.34032156196848]
  Training convolutional neural networks (CNNs) with a strict 1-Lipschitz constraint under the $l_2$ norm is useful for adversarial robustness, interpretable gradients and stable training. We introduce a procedure to certify robustness of 1-Lipschitz CNNs by replacing the last linear layer with a 1-hidden layer. We significantly advance the state-of-the-art for standard and provable robust accuracies on CIFAR-10 and CIFAR-100.
  arXiv  Detail & Related papers  (2022-11-15T19:10:12Z)
• Predicting the Stability of Hierarchical Triple Systems with Convolutional Neural Networks [68.8204255655161]
  We propose a convolutional neural network model to predict the stability of hierarchical triples. All trained models are made publicly available, allowing to predict the stability of hierarchical triple systems $200$ times faster than pure $N$-body methods.
  arXiv  Detail & Related papers  (2022-06-24T17:58:13Z)
• Scalable Lipschitz Residual Networks with Convex Potential Flows [120.27516256281359]
  We show that using convex potentials in a residual network gradient flow provides a built-in $1$-Lipschitz transformation. A comprehensive set of experiments on CIFAR-10 demonstrates the scalability of our architecture and the benefit of our approach for $ell$ provable defenses.
  arXiv  Detail & Related papers  (2021-10-25T07:12:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.