Manifold Restricted Interventional Shapley Values
- URL: http://arxiv.org/abs/2301.04041v1
- Date: Tue, 10 Jan 2023 15:47:49 GMT
- Title: Manifold Restricted Interventional Shapley Values
- Authors: Muhammad Faaiz Taufiq, Patrick Bl\"obaum, Lenon Minorics
- Abstract summary: We propose emphManifoldShap, which respects the model's domain of validity by restricting model evaluations to the data manifold.
We show, theoretically and empirically, that ManifoldShap is robust to off-manifold perturbations of the model and leads to more accurate and intuitive explanations.
- Score: 0.5156484100374059
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Shapley values are model-agnostic methods for explaining model predictions.
Many commonly used methods of computing Shapley values, known as
\emph{off-manifold methods}, rely on model evaluations on out-of-distribution
input samples. Consequently, explanations obtained are sensitive to model
behaviour outside the data distribution, which may be irrelevant for all
practical purposes. While \emph{on-manifold methods} have been proposed which
do not suffer from this problem, we show that such methods are overly dependent
on the input data distribution, and therefore result in unintuitive and
misleading explanations. To circumvent these problems, we propose
\emph{ManifoldShap}, which respects the model's domain of validity by
restricting model evaluations to the data manifold. We show, theoretically and
empirically, that ManifoldShap is robust to off-manifold perturbations of the
model and leads to more accurate and intuitive explanations than existing
state-of-the-art Shapley methods.
Related papers
- Learning Latent Space Dynamics with Model-Form Uncertainties: A Stochastic Reduced-Order Modeling Approach [0.0]
This paper presents a probabilistic approach to represent and quantify model-form uncertainties in the reduced-order modeling of complex systems.
The proposed method captures these uncertainties by expanding the approximation space through the randomization of the projection matrix.
The efficacy of the approach is assessed on canonical problems in fluid mechanics by identifying and quantifying the impact of model-form uncertainties on the inferred operators.
arXiv Detail & Related papers (2024-08-30T19:25:28Z) - Shapley Marginal Surplus for Strong Models [0.9831489366502301]
We show that while Shapley values might be accurate explainers of model predictions, machine learning models themselves are often poor explainers of the true data-generating process (DGP)
We introduce a novel variable importance algorithm, Shapley Marginal Surplus for Strong Models, that samples the space of possible models to come up with an inferential measure of feature importance.
arXiv Detail & Related papers (2024-08-16T17:06:07Z) - Diffusion models for probabilistic programming [56.47577824219207]
Diffusion Model Variational Inference (DMVI) is a novel method for automated approximate inference in probabilistic programming languages (PPLs)
DMVI is easy to implement, allows hassle-free inference in PPLs without the drawbacks of, e.g., variational inference using normalizing flows, and does not make any constraints on the underlying neural network model.
arXiv Detail & Related papers (2023-11-01T12:17:05Z) - The Implicit Delta Method [61.36121543728134]
In this paper, we propose an alternative, the implicit delta method, which works by infinitesimally regularizing the training loss of uncertainty.
We show that the change in the evaluation due to regularization is consistent for the variance of the evaluation estimator, even when the infinitesimal change is approximated by a finite difference.
arXiv Detail & Related papers (2022-11-11T19:34:17Z) - Building Reliable Explanations of Unreliable Neural Networks: Locally
Smoothing Perspective of Model Interpretation [0.0]
We present a novel method for reliably explaining the predictions of neural networks.
Our method is built on top of the assumption of smooth landscape in a loss function of the model prediction.
arXiv Detail & Related papers (2021-03-26T08:52:11Z) - Beyond Trivial Counterfactual Explanations with Diverse Valuable
Explanations [64.85696493596821]
In computer vision applications, generative counterfactual methods indicate how to perturb a model's input to change its prediction.
We propose a counterfactual method that learns a perturbation in a disentangled latent space that is constrained using a diversity-enforcing loss.
Our model improves the success rate of producing high-quality valuable explanations when compared to previous state-of-the-art methods.
arXiv Detail & Related papers (2021-03-18T12:57:34Z) - Explaining predictive models using Shapley values and non-parametric
vine copulas [2.6774008509840996]
We propose two new approaches for modelling the dependence between the features.
The performance of the proposed methods is evaluated on simulated data sets and a real data set.
Experiments demonstrate that the vine copula approaches give more accurate approximations to the true Shapley values than its competitors.
arXiv Detail & Related papers (2021-02-12T09:43:28Z) - Distilling Interpretable Models into Human-Readable Code [71.11328360614479]
Human-readability is an important and desirable standard for machine-learned model interpretability.
We propose to train interpretable models using conventional methods, and then distill them into concise, human-readable code.
We describe a piecewise-linear curve-fitting algorithm that produces high-quality results efficiently and reliably across a broad range of use cases.
arXiv Detail & Related papers (2021-01-21T01:46:36Z) - Good Classifiers are Abundant in the Interpolating Regime [64.72044662855612]
We develop a methodology to compute precisely the full distribution of test errors among interpolating classifiers.
We find that test errors tend to concentrate around a small typical value $varepsilon*$, which deviates substantially from the test error of worst-case interpolating model.
Our results show that the usual style of analysis in statistical learning theory may not be fine-grained enough to capture the good generalization performance observed in practice.
arXiv Detail & Related papers (2020-06-22T21:12:31Z) - Evaluating the Disentanglement of Deep Generative Models through
Manifold Topology [66.06153115971732]
We present a method for quantifying disentanglement that only uses the generative model.
We empirically evaluate several state-of-the-art models across multiple datasets.
arXiv Detail & Related papers (2020-06-05T20:54:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.