Related papers: Problems with Shapley-value-based explanations as feature importance measures

Problems with Shapley-value-based explanations as feature importance measures

URL: http://arxiv.org/abs/2002.11097v2
Date: Tue, 30 Jun 2020 14:38:36 GMT
Title: Problems with Shapley-value-based explanations as feature importance measures
Authors: I. Elizabeth Kumar, Suresh Venkatasubramanian, Carlos Scheidegger, Sorelle Friedler
Abstract summary: Game-theoretic formulations of feature importance have become popular as a way to "explain" machine learning models. We show that mathematical problems arise when Shapley values are used for feature importance. We argue that Shapley values do not provide explanations which suit human-centric goals of explainability.
Score: 12.08945475767566
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Game-theoretic formulations of feature importance have become popular as a way to "explain" machine learning models. These methods define a cooperative game between the features of a model and distribute influence among these input elements using some form of the game's unique Shapley values. Justification for these methods rests on two pillars: their desirable mathematical properties, and their applicability to specific motivations for explanations. We show that mathematical problems arise when Shapley values are used for feature importance and that the solutions to mitigate these necessarily induce further complexity, such as the need for causal reasoning. We also draw on additional literature to argue that Shapley values do not provide explanations which suit human-centric goals of explainability.

Related papers

Sufficient and Necessary Explanations (and What Lies in Between) [6.9035001722324685]
We study two precise notions of feature importance for general machine learning models: sufficiency and necessity. We propose a unified notion of importance that circumvents these limitations by exploring a continuum along a necessity-sufficiency axis.
arXiv Detail & Related papers (2024-09-30T15:50:57Z)
Nonparametric Partial Disentanglement via Mechanism Sparsity: Sparse Actions, Interventions and Sparse Temporal Dependencies [58.179981892921056]
This work introduces a novel principle for disentanglement we call mechanism sparsity regularization. We propose a representation learning method that induces disentanglement by simultaneously learning the latent factors. We show that the latent factors can be recovered by regularizing the learned causal graph to be sparse.
arXiv Detail & Related papers (2024-01-10T02:38:21Z)
Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
arXiv Detail & Related papers (2023-12-10T23:13:23Z)
Learning with Explanation Constraints [91.23736536228485]
We provide a learning theoretic framework to analyze how explanations can improve the learning of our models. We demonstrate the benefits of our approach over a large array of synthetic and real-world experiments.
arXiv Detail & Related papers (2023-03-25T15:06:47Z)
Human Interpretation of Saliency-based Explanation Over Text [65.29015910991261]
We study saliency-based explanations over textual data. We find that people often mis-interpret the explanations. We propose a method to adjust saliencies based on model estimates of over- and under-perception.
arXiv Detail & Related papers (2022-01-27T15:20:32Z)
Model Explanations via the Axiomatic Causal Lens [9.915489218010952]
We propose three explanation measures which aggregate the set of all but-for causes into feature importance weights. Our first measure is a natural adaptation of Chockler and Halpern's notion of causal responsibility. We extend our approach to derive a new method to compute the Shapley-Shubik and Banzhaf indices for black-box model explanations.
arXiv Detail & Related papers (2021-09-08T19:33:52Z)
Fundamental Limits and Tradeoffs in Invariant Representation Learning [99.2368462915979]
Many machine learning applications involve learning representations that achieve two competing goals. Minimax game-theoretic formulation represents a fundamental tradeoff between accuracy and invariance. We provide an information-theoretic analysis of this general and important problem under both classification and regression settings.
arXiv Detail & Related papers (2020-12-19T15:24:04Z)
Causal Shapley Values: Exploiting Causal Knowledge to Explain Individual Predictions of Complex Models [6.423239719448169]
Shapley values are designed to attribute the difference between a model's prediction and an average baseline to the different features used as input to the model. We show how these 'causal' Shapley values can be derived for general causal graphs without sacrificing any of their desirable properties.
arXiv Detail & Related papers (2020-11-03T11:11:36Z)
The Struggles of Feature-Based Explanations: Shapley Values vs. Minimal Sufficient Subsets [61.66584140190247]
We show that feature-based explanations pose problems even for explaining trivial models. We show that two popular classes of explainers, Shapley explainers and minimal sufficient subsets explainers, target fundamentally different types of ground-truth explanations.
arXiv Detail & Related papers (2020-09-23T09:45:23Z)
Explaining the data or explaining a model? Shapley values that uncover non-linear dependencies [0.0]
We introduce and demonstrate the use of the energy distance correlations, affine-invariant distance correlation, and Hilbert-Shmidt independence criterion as Shapley value characteristic functions.
arXiv Detail & Related papers (2020-07-12T15:04:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.