Sanity Checks for Explanation Uncertainty

• URL: http://arxiv.org/abs/2403.17212v1
• Date: Mon, 25 Mar 2024 21:39:33 GMT
• Title: Sanity Checks for Explanation Uncertainty
• Authors: Matias Valdenegro-Toro, Mihir Mulye,
• Abstract summary: Explanations for machine learning models can be hard to interpret or be wrong. We propose sanity checks for uncertainty explanation methods, where a weight and data randomization tests are defined for explanations with uncertainty. We experimentally show the validity and effectiveness of these tests on the CIFAR10 and California Housing datasets.
• Score: 6.9060054915724
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Explanations for machine learning models can be hard to interpret or be wrong. Combining an explanation method with an uncertainty estimation method produces explanation uncertainty. Evaluating explanation uncertainty is difficult. In this paper we propose sanity checks for uncertainty explanation methods, where a weight and data randomization tests are defined for explanations with uncertainty, allowing for quick tests to combinations of uncertainty and explanation methods. We experimentally show the validity and effectiveness of these tests on the CIFAR10 and California Housing datasets, noting that Ensembles seem to consistently pass both tests with Guided Backpropagation, Integrated Gradients, and LIME explanations.

Related papers

• How disentangled are your classification uncertainties? [6.144680854063938]
  Uncertainty Quantification in Machine Learning has progressed to predicting the source of uncertainty in a prediction. This work proposes a set of experiments to evaluate disentanglement of aleatoric and epistemic uncertainty.
  arXiv  Detail & Related papers  (2024-08-22T07:42:43Z)
• Uncertainty Quantification for Gradient-based Explanations in Neural Networks [6.9060054915724]
  We propose a pipeline to ascertain the explanation uncertainty of neural networks. We use this pipeline to produce explanation distributions for the CIFAR-10, FER+, and California Housing datasets. We compute modified pixel insertion/deletion metrics to evaluate the quality of the generated explanations.
  arXiv  Detail & Related papers  (2024-03-25T21:56:02Z)
• Identifying Drivers of Predictive Aleatoric Uncertainty [2.5311562666866494]
  We present a simple approach to explain predictive aleatoric uncertainties. We estimate uncertainty as predictive variance by adapting a neural network with a Gaussian output distribution. We quantify our findings with a nuanced benchmark analysis that includes real-world datasets.
  arXiv  Detail & Related papers  (2023-12-12T13:28:53Z)
• Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling [69.83976050879318]
  In large language models (LLMs), identifying sources of uncertainty is an important step toward improving reliability, trustworthiness, and interpretability. In this paper, we introduce an uncertainty decomposition framework for LLMs, called input clarification ensembling. Our approach generates a set of clarifications for the input, feeds them into an LLM, and ensembles the corresponding predictions.
  arXiv  Detail & Related papers  (2023-11-15T05:58:35Z)
• Shortcomings of Top-Down Randomization-Based Sanity Checks for Evaluations of Deep Neural Network Explanations [67.40641255908443]
  We identify limitations of model-randomization-based sanity checks for the purpose of evaluating explanations. Top-down model randomization preserves scales of forward pass activations with high probability.
  arXiv  Detail & Related papers  (2022-11-22T18:52:38Z)
• Uncertain Evidence in Probabilistic Models and Stochastic Simulators [80.40110074847527]
  We consider the problem of performing Bayesian inference in probabilistic models where observations are accompanied by uncertainty, referred to as uncertain evidence' We explore how to interpret uncertain evidence, and by extension the importance of proper interpretation as it pertains to inference about latent variables. We devise concrete guidelines on how to account for uncertain evidence and we provide new insights, particularly regarding consistency.
  arXiv  Detail & Related papers  (2022-10-21T20:32:59Z)
• What is Flagged in Uncertainty Quantification? Latent Density Models for Uncertainty Categorization [68.15353480798244]
  Uncertainty Quantification (UQ) is essential for creating trustworthy machine learning models. Recent years have seen a steep rise in UQ methods that can flag suspicious examples. We propose a framework for categorizing uncertain examples flagged by UQ methods in classification tasks.
  arXiv  Detail & Related papers  (2022-07-11T19:47:00Z)
• Don't Explain Noise: Robust Counterfactuals for Randomized Ensembles [50.81061839052459]
  We formalize the generation of robust counterfactual explanations as a probabilistic problem. We show the link between the robustness of ensemble models and the robustness of base learners. Our method achieves high robustness with only a small increase in the distance from counterfactual explanations to their initial observations.
  arXiv  Detail & Related papers  (2022-05-27T17:28:54Z)
• Uncertainty Estimation and Out-of-Distribution Detection for Counterfactual Explanations: Pitfalls and Solutions [7.106279650827998]
  It is often difficult to determine if the generated explanations are well grounded in the training data and sensitive to distributional shifts. This paper proposes several practical solutions that can be leveraged to solve these problems.
  arXiv  Detail & Related papers  (2021-07-20T19:09:10Z)
• Comprehensible Counterfactual Explanation on Kolmogorov-Smirnov Test [56.5373227424117]
  We tackle the problem of producing counterfactual explanations for test data failing the Kolmogorov-Smirnov (KS) test. We develop an efficient algorithm MOCHE that avoids enumerating and checking an exponential number of subsets of the test set failing the KS test.
  arXiv  Detail & Related papers  (2020-11-01T06:46:01Z)
• Getting a CLUE: A Method for Explaining Uncertainty Estimates [30.367995696223726]
  We propose a novel method for interpreting uncertainty estimates from differentiable probabilistic models. Our method, Counterfactual Latent Uncertainty Explanations (CLUE), indicates how to change an input, while keeping it on the data manifold.
  arXiv  Detail & Related papers  (2020-06-11T21:53:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.