Assessing the Reliability of Visual Explanations of Deep Models with Adversarial Perturbations

• URL: http://arxiv.org/abs/2004.10824v1
• Date: Wed, 22 Apr 2020 19:57:34 GMT
• Title: Assessing the Reliability of Visual Explanations of Deep Models with Adversarial Perturbations
• Authors: Dan Valle, Tiago Pimentel, Adriano Veloso
• Abstract summary: We propose an objective measure to evaluate the reliability of explanations of deep models. Our approach is based on changes in the network's outcome resulting from the perturbation of input images in an adversarial way. We also propose a straightforward application of our approach to clean relevance maps, creating more interpretable maps without any loss in essential explanation.
• Score: 15.067369314723958
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The interest in complex deep neural networks for computer vision applications is increasing. This leads to the need for improving the interpretable capabilities of these models. Recent explanation methods present visualizations of the relevance of pixels from input images, thus enabling the direct interpretation of properties of the input that lead to a specific output. These methods produce maps of pixel importance, which are commonly evaluated by visual inspection. This means that the effectiveness of an explanation method is assessed based on human expectation instead of actual feature importance. Thus, in this work we propose an objective measure to evaluate the reliability of explanations of deep models. Specifically, our approach is based on changes in the network's outcome resulting from the perturbation of input images in an adversarial way. We present a comparison between widely-known explanation methods using our proposed approach. Finally, we also propose a straightforward application of our approach to clean relevance maps, creating more interpretable maps without any loss in essential explanation (as per our proposed measure).

Related papers

• Automatic Discovery of Visual Circuits [66.99553804855931]
  We explore scalable methods for extracting the subgraph of a vision model's computational graph that underlies recognition of a specific visual concept. We find that our approach extracts circuits that causally affect model output, and that editing these circuits can defend large pretrained models from adversarial attacks.
  arXiv  Detail & Related papers  (2024-04-22T17:00:57Z)
• Unsupervised Interpretable Basis Extraction for Concept-Based Visual Explanations [53.973055975918655]
  We show that, intermediate layer representations become more interpretable when transformed to the bases extracted with our method. We compare the bases extracted with our method with the bases derived with a supervised approach and find that, in one aspect, the proposed unsupervised approach has a strength that constitutes a limitation of the supervised one and give potential directions for future research.
  arXiv  Detail & Related papers  (2023-03-19T00:37:19Z)
• Shap-CAM: Visual Explanations for Convolutional Neural Networks based on Shapley Value [86.69600830581912]
  We develop a novel visual explanation method called Shap-CAM based on class activation mapping. We demonstrate that Shap-CAM achieves better visual performance and fairness for interpreting the decision making process.
  arXiv  Detail & Related papers  (2022-08-07T00:59:23Z)
• ADVISE: ADaptive Feature Relevance and VISual Explanations for Convolutional Neural Networks [0.745554610293091]
  We introduce ADVISE, a new explainability method that quantifies and leverages the relevance of each unit of the feature map to provide better visual explanations. We extensively evaluate our idea in the image classification task using AlexNet, VGG16, ResNet50, and Xception pretrained on ImageNet. Our experiments further show that ADVISE fulfils the sensitivity and implementation independence axioms while passing the sanity checks.
  arXiv  Detail & Related papers  (2022-03-02T18:16:57Z)
• CAMERAS: Enhanced Resolution And Sanity preserving Class Activation Mapping for image saliency [61.40511574314069]
  Backpropagation image saliency aims at explaining model predictions by estimating model-centric importance of individual pixels in the input. We propose CAMERAS, a technique to compute high-fidelity backpropagation saliency maps without requiring any external priors.
  arXiv  Detail & Related papers  (2021-06-20T08:20:56Z)
• Revisiting The Evaluation of Class Activation Mapping for Explainability: A Novel Metric and Experimental Analysis [54.94682858474711]
  Class Activation Mapping (CAM) approaches provide an effective visualization by taking weighted averages of the activation maps. We propose a novel set of metrics to quantify explanation maps, which show better effectiveness and simplify comparisons between approaches.
  arXiv  Detail & Related papers  (2021-04-20T21:34:24Z)
• Proactive Pseudo-Intervention: Causally Informed Contrastive Learning For Interpretable Vision Models [103.64435911083432]
  We present a novel contrastive learning strategy called it Proactive Pseudo-Intervention (PPI) PPI leverages proactive interventions to guard against image features with no causal relevance. We also devise a novel causally informed salience mapping module to identify key image pixels to intervene, and show it greatly facilitates model interpretability.
  arXiv  Detail & Related papers  (2020-12-06T20:30:26Z)
• Image Super-Resolution using Explicit Perceptual Loss [17.2448277365841]
  We show how to exploit the machine learning based model which is directly trained to provide the perceptual score on generated images. The experimental results show the explicit approach has a higher perceptual score than other approaches.
  arXiv  Detail & Related papers  (2020-09-01T12:22:39Z)
• A generalizable saliency map-based interpretation of model outcome [1.14219428942199]
  We propose a non-intrusive interpretability technique that uses the input and output of the model to generate a saliency map. Experiments show that our interpretability method can reconstruct the salient part of the input with a classification accuracy of 89%.
  arXiv  Detail & Related papers  (2020-06-16T20:34:42Z)
• Uncertainty based Class Activation Maps for Visual Question Answering [30.859101872119517]
  We propose a method that obtains gradient-based certainty estimates that also provide visual attention maps. We incorporate modern probabilistic deep learning methods that we further improve by using the gradients for these estimates. The proposed technique can be thought of as a recipe for obtaining improved certainty estimates and explanations for deep learning models.
  arXiv  Detail & Related papers  (2020-01-23T19:54:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.