On Shapley Credit Allocation for Interpretability

• URL: http://arxiv.org/abs/2012.05506v1
• Date: Thu, 10 Dec 2020 08:25:32 GMT
• Title: On Shapley Credit Allocation for Interpretability
• Authors: Debraj Basu
• Abstract summary: We emphasize the importance of asking the right question when interpreting the decisions of a learning model. This paper quantifies feature relevance by weaving different natures of interpretations together with different measures as characteristic functions for Shapley symmetrization.
• Score: 1.52292571922932
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: We emphasize the importance of asking the right question when interpreting the decisions of a learning model. We discuss a natural extension of the theoretical machinery from Janzing et. al. 2020, which answers the question "Why did my model predict a person has cancer?" for answering a more involved question, "What caused my model to predict a person has cancer?" While the former quantifies the direct effects of variables on the model, the latter also accounts for indirect effects, thereby providing meaningful insights wherever human beings can reason in terms of cause and effect. We propose three broad categories for interpretations: observational, model-specific and causal each of which are significant in their own right. Furthermore, this paper quantifies feature relevance by weaving different natures of interpretations together with different measures as characteristic functions for Shapley symmetrization. Besides the widely used expected value of the model, we also discuss measures of statistical uncertainty and dispersion as informative candidates, and their merits in generating explanations for each data point, some of which are used in this context for the first time. These measures are not only useful for studying the influence of variables on the model output, but also on the predictive performance of the model, and for that we propose relevant characteristic functions that are also used for the first time.

Related papers

• Linking Model Intervention to Causal Interpretation in Model Explanation [34.21877996496178]
  We will study the conditions when an intuitive model intervention effect has a causal interpretation. This work links the model intervention effect to the causal interpretation of a model. Experiments on semi-synthetic datasets have been conducted to validate theorems and show the potential for using the model intervention effect for model interpretation.
  arXiv  Detail & Related papers  (2024-10-21T05:16:59Z)
• Nonparametric Identifiability of Causal Representations from Unknown Interventions [63.1354734978244]
  We study causal representation learning, the task of inferring latent causal variables and their causal relations from mixtures of the variables. Our goal is to identify both the ground truth latents and their causal graph up to a set of ambiguities which we show to be irresolvable from interventional data.
  arXiv  Detail & Related papers  (2023-06-01T10:51:58Z)
• Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictions [59.284907093349425]
  Large amounts of training data are one of the major reasons for the high performance of state-of-the-art NLP models. We provide a language for describing how training data influences predictions, through a causal framework. Our framework bypasses the need to retrain expensive models and allows us to estimate causal effects based on observational data alone.
  arXiv  Detail & Related papers  (2022-07-28T17:36:24Z)
• Explanatory causal effects for model agnostic explanations [27.129579550130423]
  We study the problem of estimating the contributions of features to the prediction of a specific instance by a machine learning model. A challenge is that most existing causal effects cannot be estimated from data without a known causal graph. We define an explanatory causal effect based on a hypothetical ideal experiment.
  arXiv  Detail & Related papers  (2022-06-23T08:25:31Z)
• Model Explanations via the Axiomatic Causal Lens [9.915489218010952]
  We propose three explanation measures which aggregate the set of all but-for causes into feature importance weights. Our first measure is a natural adaptation of Chockler and Halpern's notion of causal responsibility. We extend our approach to derive a new method to compute the Shapley-Shubik and Banzhaf indices for black-box model explanations.
  arXiv  Detail & Related papers  (2021-09-08T19:33:52Z)
• Latent Causal Invariant Model [128.7508609492542]
  Current supervised learning can learn spurious correlation during the data-fitting process. We propose a Latent Causal Invariance Model (LaCIM) which pursues causal prediction.
  arXiv  Detail & Related papers  (2020-11-04T10:00:27Z)
• Marginal Contribution Feature Importance -- an Axiomatic Approach for The Natural Case [13.94406695921022]
  We develop a set of axioms that represent the properties expected from a feature importance function in the natural scenario. We analyze this function for its theoretical and empirical properties and compare it to other feature importance scores.
  arXiv  Detail & Related papers  (2020-10-15T17:41:42Z)
• Double Robust Representation Learning for Counterfactual Prediction [68.78210173955001]
  We propose a novel scalable method to learn double-robust representations for counterfactual predictions. We make robust and efficient counterfactual predictions for both individual and average treatment effects. The algorithm shows competitive performance with the state-of-the-art on real world and synthetic data.
  arXiv  Detail & Related papers  (2020-10-15T16:39:26Z)
• The Struggles of Feature-Based Explanations: Shapley Values vs. Minimal Sufficient Subsets [61.66584140190247]
  We show that feature-based explanations pose problems even for explaining trivial models. We show that two popular classes of explainers, Shapley explainers and minimal sufficient subsets explainers, target fundamentally different types of ground-truth explanations.
  arXiv  Detail & Related papers  (2020-09-23T09:45:23Z)
• Nonparametric Feature Impact and Importance [0.6123324869194193]
  We give mathematical definitions of feature impact and importance, derived from partial dependence curves, that operate directly on the data. To assess quality, we show that features ranked by these definitions are competitive with existing feature selection techniques.
  arXiv  Detail & Related papers  (2020-06-08T17:07:35Z)
• CausalVAE: Structured Causal Disentanglement in Variational Autoencoder [52.139696854386976]
  The framework of variational autoencoder (VAE) is commonly used to disentangle independent factors from observations. We propose a new VAE based framework named CausalVAE, which includes a Causal Layer to transform independent factors into causal endogenous ones. Results show that the causal representations learned by CausalVAE are semantically interpretable, and their causal relationship as a Directed Acyclic Graph (DAG) is identified with good accuracy.
  arXiv  Detail & Related papers  (2020-04-18T20:09:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.