Spatial Information Bottleneck for Interpretable Visual Recognition

• URL: http://arxiv.org/abs/2511.09239v1
• Date: Thu, 13 Nov 2025 01:42:10 GMT
• Title: Spatial Information Bottleneck for Interpretable Visual Recognition
• Authors: Kaixiang Shu, Kai Meng, Junqin Luo,
• Abstract summary: Deep neural networks typically learn spatially entangled representations that conflate discriminative foreground features with spurious background correlations.<n>We propose a novel understanding framework for gradient-based attribution from an information-theoretic perspective.
• Score: 1.1533793543850381
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep neural networks typically learn spatially entangled representations that conflate discriminative foreground features with spurious background correlations, thereby undermining model interpretability and robustness. We propose a novel understanding framework for gradient-based attribution from an information-theoretic perspective. We prove that, under mild conditions, the Vector-Jacobian Products (VJP) computed during backpropagation form minimal sufficient statistics of input features with respect to class labels. Motivated by this finding, we propose an encoding-decoding perspective : forward propagation encodes inputs into class space, while VJP in backpropagation decodes this encoding back to feature space. Therefore, we propose Spatial Information Bottleneck (S-IB) to spatially disentangle information flow. By maximizing mutual information between foreground VJP and inputs while minimizing mutual information in background regions, S-IB encourages networks to encode information only in class-relevant spatial regions. Since post-hoc explanation methods fundamentally derive from VJP computations, directly optimizing VJP's spatial structure during training improves visualization quality across diverse explanation paradigms. Experiments on five benchmarks demonstrate universal improvements across six explanation methods, achieving better foreground concentration and background suppression without method-specific tuning, alongside consistent classification accuracy gains.

Related papers

• Vector Field Attention for Deformable Image Registration [9.852055065890479]
  Deformable image registration establishes non-linear spatial correspondences between fixed and moving images. Most existing deep learning-based methods require neural networks to encode location information in their feature maps. We present Vector Field Attention (VFA), a novel framework that enhances the efficiency of the existing network design by enabling direct retrieval of location correspondences.
  arXiv  Detail & Related papers  (2024-07-14T14:06:58Z)
• HiRes-LLaVA: Restoring Fragmentation Input in High-Resolution Large Vision-Language Models [96.76995840807615]
  HiRes-LLaVA is a novel framework designed to process any size of high-resolution input without altering the original contextual and geometric information. HiRes-LLaVA comprises two innovative components: (i) a SliceRestore adapter that reconstructs sliced patches into their original form, efficiently extracting both global and local features via down-up-sampling and convolution layers, and (ii) a Self-Mining Sampler to compress the vision tokens based on themselves.
  arXiv  Detail & Related papers  (2024-07-11T17:42:17Z)
• Local positional graphs and attentive local features for a data and runtime-efficient hierarchical place recognition pipeline [11.099588962062937]
  This paper proposes a runtime and data-efficient hierarchical VPR pipeline that extends existing approaches and presents novel ideas. First, we propose Local Positional Graphs (LPG), a training-free and runtime-efficient approach to encode spatial context information of local image features. Second, we present Attentive Local SPED (ATLAS), an extension of our previous local features approach with an attention module. Third, we present a hierarchical pipeline that exploits hyperdimensional computing to use the same local features as holistic HDC-descriptors for fast candidate selection and for candidate reranking.
  arXiv  Detail & Related papers  (2024-03-15T13:26:39Z)
• Towards the Uncharted: Density-Descending Feature Perturbation for Semi-supervised Semantic Segmentation [51.66997548477913]
  We propose a novel feature-level consistency learning framework named Density-Descending Feature Perturbation (DDFP) Inspired by the low-density separation assumption in semi-supervised learning, our key insight is that feature density can shed a light on the most promising direction for the segmentation classifier to explore. The proposed DDFP outperforms other designs on feature-level perturbations and shows state of the art performances on both Pascal VOC and Cityscapes dataset.
  arXiv  Detail & Related papers  (2024-03-11T06:59:05Z)
• Interpretable Spectral Variational AutoEncoder (ISVAE) for time series clustering [48.0650332513417]
  We introduce a novel model that incorporates an interpretable bottleneck-termed the Filter Bank (FB)-at the outset of a Variational Autoencoder (VAE) This arrangement compels the VAE to attend on the most informative segments of the input signal. By deliberately constraining the VAE with this FB, we promote the development of an encoding that is discernible, separable, and of reduced dimensionality.
  arXiv  Detail & Related papers  (2023-10-18T13:06:05Z)
• Improving Deep Representation Learning via Auxiliary Learnable Target Coding [69.79343510578877]
  This paper introduces a novel learnable target coding as an auxiliary regularization of deep representation learning. Specifically, a margin-based triplet loss and a correlation consistency loss on the proposed target codes are designed to encourage more discriminative representations.
  arXiv  Detail & Related papers  (2023-05-30T01:38:54Z)
• Variantional autoencoder with decremental information bottleneck for disentanglement [16.93743613675349]
  We present a novel framework for disentangled representation learning, DeVAE, which utilizes hierarchical latent spaces with decreasing information bottlenecks. The key innovation of our approach lies in connecting the hierarchical latent spaces through disentanglement-invariant transformations. We demonstrate the effectiveness of DeVAE in achieving a balance between disentanglement and reconstruction through a series of experiments and ablation studies on dSprites and Shapes3D datasets.
  arXiv  Detail & Related papers  (2023-03-22T23:38:10Z)
• Vector Quantized Wasserstein Auto-Encoder [57.29764749855623]
  We study learning deep discrete representations from the generative viewpoint. We endow discrete distributions over sequences of codewords and learn a deterministic decoder that transports the distribution over the sequences of codewords to the data distribution. We develop further theories to connect it with the clustering viewpoint of WS distance, allowing us to have a better and more controllable clustering solution.
  arXiv  Detail & Related papers  (2023-02-12T13:51:36Z)
• IntroVAC: Introspective Variational Classifiers for Learning Interpretable Latent Subspaces [6.574517227976925]
  IntroVAC learns interpretable latent subspaces by exploiting information from an additional label. We show that IntroVAC is able to learn meaningful directions in the latent space enabling fine manipulation of image attributes.
  arXiv  Detail & Related papers  (2020-08-03T10:21:41Z)
• MetaSDF: Meta-learning Signed Distance Functions [85.81290552559817]
  Generalizing across shapes with neural implicit representations amounts to learning priors over the respective function space. We formalize learning of a shape space as a meta-learning problem and leverage gradient-based meta-learning algorithms to solve this task.
  arXiv  Detail & Related papers  (2020-06-17T05:14:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.