Sanity Checks for Saliency Methods Explaining Object Detectors

• URL: http://arxiv.org/abs/2306.02424v1
• Date: Sun, 4 Jun 2023 17:57:51 GMT
• Title: Sanity Checks for Saliency Methods Explaining Object Detectors
• Authors: Deepan Chakravarthi Padmanabhan, Paul G. Pl\"oger, Octavio Arriaga, Matias Valdenegro-Toro
• Abstract summary: Saliency methods are frequently used to explain Deep Neural Network-based models. We perform sanity checks for object detection and define new qualitative criteria to evaluate the saliency explanations. We find that EfficientDet-D0 is the most interpretable method independent of the saliency method.
• Score: 5.735035463793008
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Saliency methods are frequently used to explain Deep Neural Network-based models. Adebayo et al.'s work on evaluating saliency methods for classification models illustrate certain explanation methods fail the model and data randomization tests. However, on extending the tests for various state of the art object detectors we illustrate that the ability to explain a model is more dependent on the model itself than the explanation method. We perform sanity checks for object detection and define new qualitative criteria to evaluate the saliency explanations, both for object classification and bounding box decisions, using Guided Backpropagation, Integrated Gradients, and their Smoothgrad versions, together with Faster R-CNN, SSD, and EfficientDet-D0, trained on COCO. In addition, the sensitivity of the explanation method to model parameters and data labels varies class-wise motivating to perform the sanity checks for each class. We find that EfficientDet-D0 is the most interpretable method independent of the saliency method, which passes the sanity checks with little problems.

Related papers

• DISCO: DISCovering Overfittings as Causal Rules for Text Classification Models [6.369258625916601]
  Post-hoc interpretability methods fail to capture the models' decision-making process fully. Our paper introduces DISCO, a novel method for discovering global, rule-based explanations. DISCO supports interactive explanations, enabling human inspectors to distinguish spurious causes in the rule-based output.
  arXiv  Detail & Related papers  (2024-11-07T12:12:44Z)
• COSE: A Consistency-Sensitivity Metric for Saliency on Image Classification [21.3855970055692]
  We present a set of metrics that utilize vision priors to assess the performance of saliency methods on image classification tasks. We show that although saliency methods are thought to be architecture-independent, most methods could better explain transformer-based models over convolutional-based models.
  arXiv  Detail & Related papers  (2023-09-20T01:06:44Z)
• CLIMAX: An exploration of Classifier-Based Contrastive Explanations [5.381004207943597]
  We propose a novel post-hoc model XAI technique that provides contrastive explanations justifying the classification of a black box. Our method, which we refer to as CLIMAX, is based on local classifiers. We show that we achieve better consistency as compared to baselines such as LIME, BayLIME, and SLIME.
  arXiv  Detail & Related papers  (2023-07-02T22:52:58Z)
• Evaluating the overall sensitivity of saliency-based explanation methods [1.8655840060559168]
  We address the need to generate faithful explanations of "black box" Deep Learning models. We select an existing test that is model agnostic and extend it by specifying formal thresh-olds and building criteria. We discuss the relationship between sensitivity and faithfulness and consider how the test can be adapted to assess different explanation methods in other domains.
  arXiv  Detail & Related papers  (2023-06-21T21:57:58Z)
• Zero-shot Model Diagnosis [80.36063332820568]
  A common approach to evaluate deep learning models is to build a labeled test set with attributes of interest and assess how well it performs. This paper argues the case that Zero-shot Model Diagnosis (ZOOM) is possible without the need for a test set nor labeling.
  arXiv  Detail & Related papers  (2023-03-27T17:59:33Z)
• Shortcomings of Top-Down Randomization-Based Sanity Checks for Evaluations of Deep Neural Network Explanations [67.40641255908443]
  We identify limitations of model-randomization-based sanity checks for the purpose of evaluating explanations. Top-down model randomization preserves scales of forward pass activations with high probability.
  arXiv  Detail & Related papers  (2022-11-22T18:52:38Z)
• Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanations [64.85696493596821]
  In computer vision applications, generative counterfactual methods indicate how to perturb a model's input to change its prediction. We propose a counterfactual method that learns a perturbation in a disentangled latent space that is constrained using a diversity-enforcing loss. Our model improves the success rate of producing high-quality valuable explanations when compared to previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-03-18T12:57:34Z)
• Debugging Tests for Model Explanations [18.073554618753395]
  Methods tested are able to diagnose a spurious background bug, but not conclusively identify mislabeled training examples. We complement our analysis with a human subject study, and find that subjects fail to identify defective models using attributions, but instead rely, primarily, on model predictions.
  arXiv  Detail & Related papers  (2020-11-10T22:23:25Z)
• Evaluating the Disentanglement of Deep Generative Models through Manifold Topology [66.06153115971732]
  We present a method for quantifying disentanglement that only uses the generative model. We empirically evaluate several state-of-the-art models across multiple datasets.
  arXiv  Detail & Related papers  (2020-06-05T20:54:11Z)
• Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior? [97.77183117452235]
  We carry out human subject tests to isolate the effect of algorithmic explanations on model interpretability. Clear evidence of method effectiveness is found in very few cases. Our results provide the first reliable and comprehensive estimates of how explanations influence simulatability.
  arXiv  Detail & Related papers  (2020-05-04T20:35:17Z)
• There and Back Again: Revisiting Backpropagation Saliency Methods [87.40330595283969]
  Saliency methods seek to explain the predictions of a model by producing an importance map across each input sample. A popular class of such methods is based on backpropagating a signal and analyzing the resulting gradient. We propose a single framework under which several such methods can be unified.
  arXiv  Detail & Related papers  (2020-04-06T17:58:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.