Explaining Predictive Uncertainty by Exposing Second-Order Effects

• URL: http://arxiv.org/abs/2401.17441v1
• Date: Tue, 30 Jan 2024 21:02:21 GMT
• Title: Explaining Predictive Uncertainty by Exposing Second-Order Effects
• Authors: Florian Bley and Sebastian Lapuschkin and Wojciech Samek and Gr\'egoire Montavon
• Abstract summary: We present a new method for explaining predictive uncertainty based on second-order effects. Our method is generally applicable, allowing for turning common attribution techniques into powerful second-order uncertainty explainers.
• Score: 13.83164409095901
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Explainable AI has brought transparency into complex ML blackboxes, enabling, in particular, to identify which features these models use for their predictions. So far, the question of explaining predictive uncertainty, i.e. why a model 'doubts', has been scarcely studied. Our investigation reveals that predictive uncertainty is dominated by second-order effects, involving single features or product interactions between them. We contribute a new method for explaining predictive uncertainty based on these second-order effects. Computationally, our method reduces to a simple covariance computation over a collection of first-order explanations. Our method is generally applicable, allowing for turning common attribution techniques (LRP, Gradient x Input, etc.) into powerful second-order uncertainty explainers, which we call CovLRP, CovGI, etc. The accuracy of the explanations our method produces is demonstrated through systematic quantitative evaluations, and the overall usefulness of our method is demonstrated via two practical showcases.

Related papers

• Evaluating Human Alignment and Model Faithfulness of LLM Rationale [66.75309523854476]
  We study how well large language models (LLMs) explain their generations through rationales. We show that prompting-based methods are less "faithful" than attribution-based explanations.
  arXiv  Detail & Related papers  (2024-06-28T20:06:30Z)
• Cycles of Thought: Measuring LLM Confidence through Stable Explanations [53.15438489398938]
  Large language models (LLMs) can reach and even surpass human-level accuracy on a variety of benchmarks, but their overconfidence in incorrect responses is still a well-documented failure mode. We propose a framework for measuring an LLM's uncertainty with respect to the distribution of generated explanations for an answer.
  arXiv  Detail & Related papers  (2024-06-05T16:35:30Z)
• Selective Explanations [14.312717332216073]
  A machine learning model is trained to predict feature attribution scores with only one inference. Despite their efficiency, amortized explainers can produce inaccurate predictions and misleading explanations. We propose selective explanations, a novel feature attribution method that detects when amortized explainers generate low-quality explanations.
  arXiv  Detail & Related papers  (2024-05-29T23:08:31Z)
• Identifying Drivers of Predictive Aleatoric Uncertainty [2.5311562666866494]
  We present a simple approach to explain predictive aleatoric uncertainties. We estimate uncertainty as predictive variance by adapting a neural network with a Gaussian output distribution. We quantify our findings with a nuanced benchmark analysis that includes real-world datasets.
  arXiv  Detail & Related papers  (2023-12-12T13:28:53Z)
• Explaining Explainability: Towards Deeper Actionable Insights into Deep Learning through Second-order Explainability [70.60433013657693]
  Second-order explainable AI (SOXAI) was recently proposed to extend explainable AI (XAI) from the instance level to the dataset level. We demonstrate for the first time, via example classification and segmentation cases, that eliminating irrelevant concepts from the training set based on actionable insights from SOXAI can enhance a model's performance.
  arXiv  Detail & Related papers  (2023-06-14T23:24:01Z)
• Boundary-Aware Uncertainty for Feature Attribution Explainers [4.2130431095114895]
  We propose a unified uncertainty estimate combining decision boundary-aware uncertainty with explanation function approximation uncertainty. We show theoretically that the proposed kernel similarity increases with decision boundary complexity. Empirical results on multiple datasets show that the GPEC uncertainty estimate improves understanding of explanations as compared to existing methods.
  arXiv  Detail & Related papers  (2022-10-05T17:37:14Z)
• Don't Explain Noise: Robust Counterfactuals for Randomized Ensembles [50.81061839052459]
  We formalize the generation of robust counterfactual explanations as a probabilistic problem. We show the link between the robustness of ensemble models and the robustness of base learners. Our method achieves high robustness with only a small increase in the distance from counterfactual explanations to their initial observations.
  arXiv  Detail & Related papers  (2022-05-27T17:28:54Z)
• On the Benefits of Large Learning Rates for Kernel Methods [110.03020563291788]
  We show that a phenomenon can be precisely characterized in the context of kernel methods. We consider the minimization of a quadratic objective in a separable Hilbert space, and show that with early stopping, the choice of learning rate influences the spectral decomposition of the obtained solution.
  arXiv  Detail & Related papers  (2022-02-28T13:01:04Z)
• Deep learning: a statistical viewpoint [120.94133818355645]
  Deep learning has revealed some major surprises from a theoretical perspective. In particular, simple gradient methods easily find near-perfect solutions to non-optimal training problems. We conjecture that specific principles underlie these phenomena.
  arXiv  Detail & Related papers  (2021-03-16T16:26:36Z)
• Towards the Unification and Robustness of Perturbation and Gradient Based Explanations [23.41512277145231]
  We analyze two popular post hoc interpretation techniques: SmoothGrad which is a gradient based method, and a variant of LIME which is a perturbation based method. We derive explicit closed form expressions for the explanations output by these two methods and show that they both converge to the same explanation in expectation. We empirically validate our theory using extensive experimentation on both synthetic and real world datasets.
  arXiv  Detail & Related papers  (2021-02-21T14:51:18Z)
• Getting a CLUE: A Method for Explaining Uncertainty Estimates [30.367995696223726]
  We propose a novel method for interpreting uncertainty estimates from differentiable probabilistic models. Our method, Counterfactual Latent Uncertainty Explanations (CLUE), indicates how to change an input, while keeping it on the data manifold.
  arXiv  Detail & Related papers  (2020-06-11T21:53:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.