How Well Do Feature-Additive Explainers Explain Feature-Additive Predictors?

• URL: http://arxiv.org/abs/2310.18496v1
• Date: Fri, 27 Oct 2023 21:16:28 GMT
• Title: How Well Do Feature-Additive Explainers Explain Feature-Additive Predictors?
• Authors: Zachariah Carmichael, Walter J. Scheirer
• Abstract summary: We ask the question: can popular feature-additive explainers (e.g., LIME, SHAP, SHAPR, MAPLE, and PDP) explain feature-additive predictors? Herein, we evaluate such explainers on ground truth that is analytically derived from the additive structure of a model. Our results suggest that all explainers eventually fail to correctly attribute the importance of features, especially when a decision-making process involves feature interactions.
• Score: 12.993027779814478
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Surging interest in deep learning from high-stakes domains has precipitated concern over the inscrutable nature of black box neural networks. Explainable AI (XAI) research has led to an abundance of explanation algorithms for these black boxes. Such post hoc explainers produce human-comprehensible explanations, however, their fidelity with respect to the model is not well understood - explanation evaluation remains one of the most challenging issues in XAI. In this paper, we ask a targeted but important question: can popular feature-additive explainers (e.g., LIME, SHAP, SHAPR, MAPLE, and PDP) explain feature-additive predictors? Herein, we evaluate such explainers on ground truth that is analytically derived from the additive structure of a model. We demonstrate the efficacy of our approach in understanding these explainers applied to symbolic expressions, neural networks, and generalized additive models on thousands of synthetic and several real-world tasks. Our results suggest that all explainers eventually fail to correctly attribute the importance of features, especially when a decision-making process involves feature interactions.

Related papers

• T-Explainer: A Model-Agnostic Explainability Framework Based on Gradients [5.946429628497358]
  We introduce T-Explainer, a novel local additive attribution explainer based on Taylor expansion. It has desirable properties, such as local accuracy and consistency, making T-Explainer stable over multiple runs.
  arXiv  Detail & Related papers  (2024-04-25T10:40:49Z)
• Gradient based Feature Attribution in Explainable AI: A Technical Review [13.848675695545909]
  Surge in black-box AI models has prompted the need to explain the internal mechanism and justify their reliability. gradient based explanations can be directly adopted for neural network models. We introduce both human and quantitative evaluations to measure algorithm performance.
  arXiv  Detail & Related papers  (2024-03-15T15:49:31Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• Explaining Explainability: Towards Deeper Actionable Insights into Deep Learning through Second-order Explainability [70.60433013657693]
  Second-order explainable AI (SOXAI) was recently proposed to extend explainable AI (XAI) from the instance level to the dataset level. We demonstrate for the first time, via example classification and segmentation cases, that eliminating irrelevant concepts from the training set based on actionable insights from SOXAI can enhance a model's performance.
  arXiv  Detail & Related papers  (2023-06-14T23:24:01Z)
• Behave-XAI: Deep Explainable Learning of Behavioral Representational Data [0.0]
  We use explainable or human understandable AI for a behavioral mining scenario. We first formulate the behavioral mining problem in deep convolutional neural network architecture. Once the model is developed, explanations are presented in front of users.
  arXiv  Detail & Related papers  (2022-12-30T18:08:48Z)
• Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces [14.70409833767752]
  Explainable AI aims to overcome the black-box nature of complex ML models like neural networks by generating explanations for their predictions. We propose two new analyses, extending principles found in PCA or ICA to explanations. These novel analyses, which we call principal relevant component analysis (PRCA) and disentangled relevant subspace analysis (DRSA), maximize relevance instead of e.g. variance or kurtosis.
  arXiv  Detail & Related papers  (2022-12-30T18:04:25Z)
• Human Interpretation of Saliency-based Explanation Over Text [65.29015910991261]
  We study saliency-based explanations over textual data. We find that people often mis-interpret the explanations. We propose a method to adjust saliencies based on model estimates of over- and under-perception.
  arXiv  Detail & Related papers  (2022-01-27T15:20:32Z)
• CX-ToM: Counterfactual Explanations with Theory-of-Mind for Enhancing Human Trust in Image Recognition Models [84.32751938563426]
  We propose a new explainable AI (XAI) framework for explaining decisions made by a deep convolutional neural network (CNN) In contrast to the current methods in XAI that generate explanations as a single shot response, we pose explanation as an iterative communication process. Our framework generates sequence of explanations in a dialog by mediating the differences between the minds of machine and human user.
  arXiv  Detail & Related papers  (2021-09-03T09:46:20Z)
• The Struggles of Feature-Based Explanations: Shapley Values vs. Minimal Sufficient Subsets [61.66584140190247]
  We show that feature-based explanations pose problems even for explaining trivial models. We show that two popular classes of explainers, Shapley explainers and minimal sufficient subsets explainers, target fundamentally different types of ground-truth explanations.
  arXiv  Detail & Related papers  (2020-09-23T09:45:23Z)
• Explainability in Deep Reinforcement Learning [68.8204255655161]
  We review recent works in the direction to attain Explainable Reinforcement Learning (XRL) In critical situations where it is essential to justify and explain the agent's behaviour, better explainability and interpretability of RL models could help gain scientific insight on the inner workings of what is still considered a black box.
  arXiv  Detail & Related papers  (2020-08-15T10:11:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.