Evaluating Input Perturbation Methods for Interpreting CNNs and Saliency Map Comparison

• URL: http://arxiv.org/abs/2101.10977v1
• Date: Tue, 26 Jan 2021 18:11:06 GMT
• Title: Evaluating Input Perturbation Methods for Interpreting CNNs and Saliency Map Comparison
• Authors: Lukas Brunke, Prateek Agrawal, Nikhil George
• Abstract summary: In this paper we show that arguably neutral baseline images still impact the generated saliency maps and their evaluation with input perturbations. We experimentally reveal inconsistencies among a selection of input perturbation methods and find that they lack robustness for generating saliency maps and for evaluating saliency maps as saliency metrics.
• Score: 9.023847175654602
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Input perturbation methods occlude parts of an input to a function and measure the change in the function's output. Recently, input perturbation methods have been applied to generate and evaluate saliency maps from convolutional neural networks. In practice, neutral baseline images are used for the occlusion, such that the baseline image's impact on the classification probability is minimal. However, in this paper we show that arguably neutral baseline images still impact the generated saliency maps and their evaluation with input perturbations. We also demonstrate that many choices of hyperparameters lead to the divergence of saliency maps generated by input perturbations. We experimentally reveal inconsistencies among a selection of input perturbation methods and find that they lack robustness for generating saliency maps and for evaluating saliency maps as saliency metrics.

Related papers

• Unlearning-based Neural Interpretations [51.99182464831169]
  We show that current baselines defined using static functions are biased, fragile and manipulable. We propose UNI to compute an (un)learnable, debiased and adaptive baseline by perturbing the input towards an unlearning direction of steepest ascent.
  arXiv  Detail & Related papers  (2024-10-10T16:02:39Z)
• Smoothed Embeddings for Certified Few-Shot Learning [63.68667303948808]
  We extend randomized smoothing to few-shot learning models that map inputs to normalized embeddings. Our results are confirmed by experiments on different datasets.
  arXiv  Detail & Related papers  (2022-02-02T18:19:04Z)
• Where do Models go Wrong? Parameter-Space Saliency Maps for Explainability [47.18202269163001]
  We take a different approach to saliency, in which we identify and analyze the network parameters, rather than inputs. We find that samples which cause similar parameters to malfunction are semantically similar. We also show that pruning the most salient parameters for a wrongly classified sample often improves model behavior.
  arXiv  Detail & Related papers  (2021-08-03T07:32:34Z)
• CAMERAS: Enhanced Resolution And Sanity preserving Class Activation Mapping for image saliency [61.40511574314069]
  Backpropagation image saliency aims at explaining model predictions by estimating model-centric importance of individual pixels in the input. We propose CAMERAS, a technique to compute high-fidelity backpropagation saliency maps without requiring any external priors.
  arXiv  Detail & Related papers  (2021-06-20T08:20:56Z)
• Investigating sanity checks for saliency maps with image and text classification [1.836681984330549]
  Saliency maps have shown to be both useful and misleading for explaining model predictions especially in the context of images. We analyze the effects of the input multiplier in certain saliency maps using similarity scores, max-sensitivity and infidelity evaluation metrics.
  arXiv  Detail & Related papers  (2021-06-08T23:23:42Z)
• Input Bias in Rectified Gradients and Modified Saliency Maps [0.0]
  Saliency maps provide an intuitive way to identify input features with substantial influences on classifications or latent concepts. Several modifications to conventional saliency maps, such as Rectified Gradients, have been introduced to allegedly denoise and improve interpretability. We demonstrate that dark areas of an input image are not highlighted by a saliency map using Rectified Gradients, even if it is relevant for the class or concept.
  arXiv  Detail & Related papers  (2020-11-10T09:45:13Z)
• Understanding Integrated Gradients with SmoothTaylor for Deep Neural Network Attribution [70.78655569298923]
  Integrated Gradients as an attribution method for deep neural network models offers simple implementability. It suffers from noisiness of explanations which affects the ease of interpretability. The SmoothGrad technique is proposed to solve the noisiness issue and smoothen the attribution maps of any gradient-based attribution method.
  arXiv  Detail & Related papers  (2020-04-22T10:43:19Z)
• Embedding Propagation: Smoother Manifold for Few-Shot Classification [131.81692677836202]
  We propose to use embedding propagation as an unsupervised non-parametric regularizer for manifold smoothing in few-shot classification. We empirically show that embedding propagation yields a smoother embedding manifold. We show that embedding propagation consistently improves the accuracy of the models in multiple semi-supervised learning scenarios by up to 16% points.
  arXiv  Detail & Related papers  (2020-03-09T13:51:09Z)
• DANCE: Enhancing saliency maps using decoys [35.46266461621123]
  We propose a framework that improves the robustness of saliency methods by following a two-step procedure. First, we introduce a perturbation mechanism that subtly varies the input sample without changing its intermediate representations. Second, we compute saliency maps for perturbed samples and propose a new method to aggregate saliency maps.
  arXiv  Detail & Related papers  (2020-02-03T01:21:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.