A Super-pixel-based Approach to the Stable Interpretation of Neural Networks

• URL: http://arxiv.org/abs/2412.14509v1
• Date: Thu, 19 Dec 2024 04:17:32 GMT
• Title: A Super-pixel-based Approach to the Stable Interpretation of Neural Networks
• Authors: Shizhan Gong, Jingwei Zhang, Qi Dou, Farzan Farnia,
• Abstract summary: We propose a novel pixel strategy to boost the stability and generalizability of gradient-based saliency maps. We show that the grouping of pixels reduces the variance of the saliency map and improves the generalization behavior of the interpretation method.
• Score: 20.252282961052945
• License:
• Abstract: Saliency maps are widely used in the computer vision community for interpreting neural network classifiers. However, due to the randomness of training samples and optimization algorithms, the resulting saliency maps suffer from a significant level of stochasticity, making it difficult for domain experts to capture the intrinsic factors that influence the neural network's decision. In this work, we propose a novel pixel partitioning strategy to boost the stability and generalizability of gradient-based saliency maps. Through both theoretical analysis and numerical experiments, we demonstrate that the grouping of pixels reduces the variance of the saliency map and improves the generalization behavior of the interpretation method. Furthermore, we propose a sensible grouping strategy based on super-pixels which cluster pixels into groups that align well with the semantic meaning of the images. We perform several numerical experiments on CIFAR-10 and ImageNet. Our empirical results suggest that the super-pixel-based interpretation maps consistently improve the stability and quality over the pixel-based saliency maps.

Related papers

• PCIM: Learning Pixel Attributions via Pixel-wise Channel Isolation Mixing in High Content Imaging [1.1866227238721938]
  This work introduces a novel method, Pixel-wise Channel Isolation Mixing ( PCIM), to calculate pixel attribution maps. PCIM treats each pixel as a distinct input channel and trains a blending layer to mix these pixels, reflecting specific classifications. This unique approach allows the generation of pixel attribution maps for each image, but agnostic to the choice of the underlying classification network.
  arXiv  Detail & Related papers  (2024-12-03T08:48:30Z)
• Pixel-Inconsistency Modeling for Image Manipulation Localization [59.968362815126326]
  Digital image forensics plays a crucial role in image authentication and manipulation localization. This paper presents a generalized and robust manipulation localization model through the analysis of pixel inconsistency artifacts. Experiments show that our method successfully extracts inherent pixel-inconsistency forgery fingerprints.
  arXiv  Detail & Related papers  (2023-09-30T02:54:51Z)
• Single-Image Super-Resolution Reconstruction based on the Differences of Neighboring Pixels [3.257500143434429]
  The deep learning technique was used to increase the performance of single image super-resolution (SISR) In this paper, we propose the differences of neighboring pixels to regularize the CNN by constructing a graph from the estimated image and the ground-truth image. The proposed method outperforms the state-of-the-art methods in terms of quantitative and qualitative evaluation of the benchmark datasets.
  arXiv  Detail & Related papers  (2022-12-28T07:30:07Z)
• Probabilistic Deep Metric Learning for Hyperspectral Image Classification [91.5747859691553]
  This paper proposes a probabilistic deep metric learning framework for hyperspectral image classification. It aims to predict the category of each pixel for an image captured by hyperspectral sensors. Our framework can be readily applied to existing hyperspectral image classification methods.
  arXiv  Detail & Related papers  (2022-11-15T17:57:12Z)
• Shap-CAM: Visual Explanations for Convolutional Neural Networks based on Shapley Value [86.69600830581912]
  We develop a novel visual explanation method called Shap-CAM based on class activation mapping. We demonstrate that Shap-CAM achieves better visual performance and fairness for interpreting the decision making process.
  arXiv  Detail & Related papers  (2022-08-07T00:59:23Z)
• Deep Semantic Statistics Matching (D2SM) Denoising Network [70.01091467628068]
  We introduce the Deep Semantic Statistics Matching (D2SM) Denoising Network. It exploits semantic features of pretrained classification networks, then it implicitly matches the probabilistic distribution of clear images at the semantic feature space. By learning to preserve the semantic distribution of denoised images, we empirically find our method significantly improves the denoising capabilities of networks.
  arXiv  Detail & Related papers  (2022-07-19T14:35:42Z)
• Rethinking Unsupervised Neural Superpixel Segmentation [6.123324869194195]
  unsupervised learning for superpixel segmentation via CNNs has been studied. We propose three key elements to improve the efficacy of such networks. By experimenting with the BSDS500 dataset, we find evidence to the significance of our proposal.
  arXiv  Detail & Related papers  (2022-06-21T09:30:26Z)
• Sharp-GAN: Sharpness Loss Regularized GAN for Histopathology Image Synthesis [65.47507533905188]
  Conditional generative adversarial networks have been applied to generate synthetic histopathology images. We propose a sharpness loss regularized generative adversarial network to synthesize realistic histopathology images.
  arXiv  Detail & Related papers  (2021-10-27T18:54:25Z)
• Gigapixel Histopathological Image Analysis using Attention-based Neural Networks [7.1715252990097325]
  We propose a CNN structure consisting of a compressing path and a learning path. Our method integrates both global and local information, is flexible with regard to the size of the input images and only requires weak image-level labels.
  arXiv  Detail & Related papers  (2021-01-25T10:18:52Z)
• Probabilistic Graph Attention Network with Conditional Kernels for Pixel-Wise Prediction [158.88345945211185]
  We present a novel approach that advances the state of the art on pixel-level prediction in a fundamental aspect, i.e. structured multi-scale features learning and fusion. We propose a probabilistic graph attention network structure based on a novel Attention-Gated Conditional Random Fields (AG-CRFs) model for learning and fusing multi-scale representations in a principled manner.
  arXiv  Detail & Related papers  (2021-01-08T04:14:29Z)
• Probabilistic Semantic Segmentation Refinement by Monte Carlo Region Growing [0.7424262881242935]
  We introduce a fully unsupervised post-processing algorithm that exploits Monte Carlo sampling and pixel similarities to propagate high-confidence pixel labels into regions of low-confidence classification. Experiments using multiple modern semantic segmentation networks and benchmark datasets demonstrate the effectiveness of our approach for the refinement of segmentation predictions at different levels of coarseness.
  arXiv  Detail & Related papers  (2020-05-12T15:23:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.