Problems with Shapley-value-based explanations as feature importance measures

• URL: http://arxiv.org/abs/2002.11097v2
• Date: Tue, 30 Jun 2020 14:38:36 GMT
• Title: Problems with Shapley-value-based explanations as feature importance measures
• Authors: I. Elizabeth Kumar, Suresh Venkatasubramanian, Carlos Scheidegger, Sorelle Friedler
• Abstract summary: Game-theoretic formulations of feature importance have become popular as a way to "explain" machine learning models. We show that mathematical problems arise when Shapley values are used for feature importance. We argue that Shapley values do not provide explanations which suit human-centric goals of explainability.
• Score: 12.08945475767566
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Game-theoretic formulations of feature importance have become popular as a way to "explain" machine learning models. These methods define a cooperative game between the features of a model and distribute influence among these input elements using some form of the game's unique Shapley values. Justification for these methods rests on two pillars: their desirable mathematical properties, and their applicability to specific motivations for explanations. We show that mathematical problems arise when Shapley values are used for feature importance and that the solutions to mitigate these necessarily induce further complexity, such as the need for causal reasoning. We also draw on additional literature to argue that Shapley values do not provide explanations which suit human-centric goals of explainability.

Related papers

• A Critical Assessment of Interpretable and Explainable Machine Learning for Intrusion Detection [0.0]
  We study the use of overly complex and opaque ML models, unaccounted data imbalances and correlated features, inconsistent influential features across different explanation methods, and the implausible utility of explanations. Specifically, we advise avoiding complex opaque models such as Deep Neural Networks and instead using interpretable ML models such as Decision Trees. We find that feature-based model explanations are most often inconsistent across different settings.
  arXiv  Detail & Related papers  (2024-07-04T15:35:42Z)
• Nonparametric Partial Disentanglement via Mechanism Sparsity: Sparse Actions, Interventions and Sparse Temporal Dependencies [58.179981892921056]
  This work introduces a novel principle for disentanglement we call mechanism sparsity regularization. We propose a representation learning method that induces disentanglement by simultaneously learning the latent factors. We show that the latent factors can be recovered by regularizing the learned causal graph to be sparse.
  arXiv  Detail & Related papers  (2024-01-10T02:38:21Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• Learning with Explanation Constraints [91.23736536228485]
  We provide a learning theoretic framework to analyze how explanations can improve the learning of our models. We demonstrate the benefits of our approach over a large array of synthetic and real-world experiments.
  arXiv  Detail & Related papers  (2023-03-25T15:06:47Z)
• Human Interpretation of Saliency-based Explanation Over Text [65.29015910991261]
  We study saliency-based explanations over textual data. We find that people often mis-interpret the explanations. We propose a method to adjust saliencies based on model estimates of over- and under-perception.
  arXiv  Detail & Related papers  (2022-01-27T15:20:32Z)
• Model Explanations via the Axiomatic Causal Lens [9.915489218010952]
  We propose three explanation measures which aggregate the set of all but-for causes into feature importance weights. Our first measure is a natural adaptation of Chockler and Halpern's notion of causal responsibility. We extend our approach to derive a new method to compute the Shapley-Shubik and Banzhaf indices for black-box model explanations.
  arXiv  Detail & Related papers  (2021-09-08T19:33:52Z)
• Fundamental Limits and Tradeoffs in Invariant Representation Learning [99.2368462915979]
  Many machine learning applications involve learning representations that achieve two competing goals. Minimax game-theoretic formulation represents a fundamental tradeoff between accuracy and invariance. We provide an information-theoretic analysis of this general and important problem under both classification and regression settings.
  arXiv  Detail & Related papers  (2020-12-19T15:24:04Z)
• Causal Shapley Values: Exploiting Causal Knowledge to Explain Individual Predictions of Complex Models [6.423239719448169]
  Shapley values are designed to attribute the difference between a model's prediction and an average baseline to the different features used as input to the model. We show how these 'causal' Shapley values can be derived for general causal graphs without sacrificing any of their desirable properties.
  arXiv  Detail & Related papers  (2020-11-03T11:11:36Z)
• The Struggles of Feature-Based Explanations: Shapley Values vs. Minimal Sufficient Subsets [61.66584140190247]
  We show that feature-based explanations pose problems even for explaining trivial models. We show that two popular classes of explainers, Shapley explainers and minimal sufficient subsets explainers, target fundamentally different types of ground-truth explanations.
  arXiv  Detail & Related papers  (2020-09-23T09:45:23Z)
• Explaining the data or explaining a model? Shapley values that uncover non-linear dependencies [0.0]
  We introduce and demonstrate the use of the energy distance correlations, affine-invariant distance correlation, and Hilbert-Shmidt independence criterion as Shapley value characteristic functions.
  arXiv  Detail & Related papers  (2020-07-12T15:04:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.