Do Sparse Subnetworks Exhibit Cognitively Aligned Attention? Effects of Pruning on Saliency Map Fidelity, Sparsity, and Concept Coherence

• URL: http://arxiv.org/abs/2509.21387v3
• Date: Sun, 05 Oct 2025 20:06:23 GMT
• Title: Do Sparse Subnetworks Exhibit Cognitively Aligned Attention? Effects of Pruning on Saliency Map Fidelity, Sparsity, and Concept Coherence
• Abstract summary: We investigate how magnitude-based pruning followed by fine-tuning affects both low-level saliency maps and high-level concept representations.<n>Our results show that light-to-moderate pruning improves saliency-map focus and faithfulness while retaining distinct, semantically meaningful concepts.<n>In contrast, aggressive pruning merges heterogeneous features, reducing saliency map sparsity and concept coherence despite maintaining accuracy.
• Score: 2.7415651415305597
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Prior works have shown that neural networks can be heavily pruned while preserving performance, but the impact of pruning on model interpretability remains unclear. In this work, we investigate how magnitude-based pruning followed by fine-tuning affects both low-level saliency maps and high-level concept representations. Using a ResNet-18 trained on ImageNette, we compare post-hoc explanations from Vanilla Gradients (VG) and Integrated Gradients (IG) across pruning levels, evaluating sparsity and faithfulness. We further apply CRAFT-based concept extraction to track changes in semantic coherence of learned concepts. Our results show that light-to-moderate pruning improves saliency-map focus and faithfulness while retaining distinct, semantically meaningful concepts. In contrast, aggressive pruning merges heterogeneous features, reducing saliency map sparsity and concept coherence despite maintaining accuracy. These findings suggest that while pruning can shape internal representations toward more human-aligned attention patterns, excessive pruning undermines interpretability.

Related papers

• When Pruning Meets Interpretability: Preserving Sparse Autoencoder Robustness in LLMs [15.418870425179875]
  Sparse autoencoders (SAEs) are widely used to interpret the internal representations of large language models.<n>We show that for a fixed SAE, its impact is governed by perturbation energy.<n>We propose a layer-wise sparsity allocation strategy, achieving lower perplexity under the same average pruning sparsity.
  arXiv  Detail & Related papers  (2026-08-26T15:57:42Z)
• SaliencyDecor: Enhancing Neural Network Interpretability through Feature Decorrelation [4.096797783087454]
  SaliencyDecor is a training framework that enforces feature decorrelation to improve attribution fidelity without modifying saliency methods or model architectures.<n>We show that our method produces substantially sharper and more object-focused saliency maps while simultaneously improving predictive performance.
  arXiv  Detail & Related papers  (2026-04-28T07:25:07Z)
• Pulling Back the Curtain on ReLU Networks [0.0]
  gradients of deeper neurons are notoriously misaligned, which obscures the network's internal representations.<n>We prove this by applying soft gating in the backward pass only, reducing the local impact of weakly excited neurons.<n>The resulting modified gradients, which we call "excitation pullbacks", exhibit striking perceptual alignment.
  arXiv  Detail & Related papers  (2025-07-30T16:47:42Z)
• Visual-TCAV: Concept-based Attribution and Saliency Maps for Post-hoc Explainability in Image Classification [3.9626211140865464]
  Convolutional Neural Networks (CNNs) have seen significant performance improvements in recent years.<n>However, due to their size and complexity, they function as black-boxes, leading to transparency concerns.<n>This paper introduces a novel post-hoc explainability framework, Visual-TCAV, which aims to bridge the gap between these methods.
  arXiv  Detail & Related papers  (2024-11-08T16:52:52Z)
• Improving Network Interpretability via Explanation Consistency Evaluation [56.14036428778861]
  We propose a framework that acquires more explainable activation heatmaps and simultaneously increase the model performance. Specifically, our framework introduces a new metric, i.e., explanation consistency, to reweight the training samples adaptively in model learning. Our framework then promotes the model learning by paying closer attention to those training samples with a high difference in explanations.
  arXiv  Detail & Related papers  (2024-08-08T17:20:08Z)
• A Learning Paradigm for Interpretable Gradients [9.074325843851726]
  We present a novel training approach to improve the quality of gradients for interpretability. We find that the resulting gradient is qualitatively less noisy and improves quantitatively the interpretability properties of different networks.
  arXiv  Detail & Related papers  (2024-04-23T13:32:29Z)
• Theoretical Characterization of How Neural Network Pruning Affects its Generalization [131.1347309639727]
  This work makes the first attempt to study how different pruning fractions affect the model's gradient descent dynamics and generalization. It is shown that as long as the pruning fraction is below a certain threshold, gradient descent can drive the training loss toward zero. More surprisingly, the generalization bound gets better as the pruning fraction gets larger.
  arXiv  Detail & Related papers  (2023-01-01T03:10:45Z)
• The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training [111.15069968583042]
  Random pruning is arguably the most naive way to attain sparsity in neural networks, but has been deemed uncompetitive by either post-training pruning or sparse training. We empirically demonstrate that sparsely training a randomly pruned network from scratch can match the performance of its dense equivalent. Our results strongly suggest there is larger-than-expected room for sparse training at scale, and the benefits of sparsity might be more universal beyond carefully designed pruning.
  arXiv  Detail & Related papers  (2022-02-05T21:19:41Z)
• TSG: Target-Selective Gradient Backprop for Probing CNN Visual Saliency [72.9106103283475]
  We study the visual saliency, a.k.a. visual explanation, to interpret convolutional neural networks. Inspired by those observations, we propose a novel visual saliency framework, termed Target-Selective Gradient (TSG) backprop. The proposed TSG consists of two components, namely, TSG-Conv and TSG-FC, which rectify the gradients for convolutional layers and fully-connected layers, respectively.
  arXiv  Detail & Related papers  (2021-10-11T12:00:20Z)
• CAMERAS: Enhanced Resolution And Sanity preserving Class Activation Mapping for image saliency [61.40511574314069]
  Backpropagation image saliency aims at explaining model predictions by estimating model-centric importance of individual pixels in the input. We propose CAMERAS, a technique to compute high-fidelity backpropagation saliency maps without requiring any external priors.
  arXiv  Detail & Related papers  (2021-06-20T08:20:56Z)
• Sparse Training via Boosting Pruning Plasticity with Neuroregeneration [79.78184026678659]
  We study the effect of pruning throughout training from the perspective of pruning plasticity. We design a novel gradual magnitude pruning (GMP) method, named gradual pruning with zero-cost neuroregeneration (GraNet) and its dynamic sparse training (DST) variant (GraNet-ST) Perhaps most impressively, the latter for the first time boosts the sparse-to-sparse training performance over various dense-to-sparse methods by a large margin with ResNet-50 on ImageNet.
  arXiv  Detail & Related papers  (2021-06-19T02:09:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.