On Minimizing the Impact of Dataset Shifts on Actionable Explanations
- URL: http://arxiv.org/abs/2306.06716v1
- Date: Sun, 11 Jun 2023 16:34:19 GMT
- Title: On Minimizing the Impact of Dataset Shifts on Actionable Explanations
- Authors: Anna P. Meyer, Dan Ley, Suraj Srinivas, Himabindu Lakkaraju
- Abstract summary: We conduct rigorous theoretical analysis to demonstrate that model curvature, weight decay parameters while training, and the magnitude of the dataset shift are key factors that determine the extent of explanation (in)stability.
- Score: 14.83940426256441
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The Right to Explanation is an important regulatory principle that allows
individuals to request actionable explanations for algorithmic decisions.
However, several technical challenges arise when providing such actionable
explanations in practice. For instance, models are periodically retrained to
handle dataset shifts. This process may invalidate some of the previously
prescribed explanations, thus rendering them unactionable. But, it is unclear
if and when such invalidations occur, and what factors determine explanation
stability i.e., if an explanation remains unchanged amidst model retraining due
to dataset shifts. In this paper, we address the aforementioned gaps and
provide one of the first theoretical and empirical characterizations of the
factors influencing explanation stability. To this end, we conduct rigorous
theoretical analysis to demonstrate that model curvature, weight decay
parameters while training, and the magnitude of the dataset shift are key
factors that determine the extent of explanation (in)stability. Extensive
experimentation with real-world datasets not only validates our theoretical
results, but also demonstrates that the aforementioned factors dramatically
impact the stability of explanations produced by various state-of-the-art
methods.
Related papers
- Identifiable Latent Neural Causal Models [82.14087963690561]
Causal representation learning seeks to uncover latent, high-level causal representations from low-level observed data.
We determine the types of distribution shifts that do contribute to the identifiability of causal representations.
We translate our findings into a practical algorithm, allowing for the acquisition of reliable latent causal representations.
arXiv Detail & Related papers (2024-03-23T04:13:55Z) - Understanding Disparities in Post Hoc Machine Learning Explanation [2.965442487094603]
Previous work has highlighted that existing post-hoc explanation methods exhibit disparities in explanation fidelity (across 'race' and 'gender' as sensitive attributes)
We specifically assess challenges to explanation disparities that originate from properties of the data.
Results indicate that disparities in model explanations can also depend on data and model properties.
arXiv Detail & Related papers (2024-01-25T22:09:28Z) - Nonparametric Partial Disentanglement via Mechanism Sparsity: Sparse
Actions, Interventions and Sparse Temporal Dependencies [58.179981892921056]
This work introduces a novel principle for disentanglement we call mechanism sparsity regularization.
We propose a representation learning method that induces disentanglement by simultaneously learning the latent factors.
We show that the latent factors can be recovered by regularizing the learned causal graph to be sparse.
arXiv Detail & Related papers (2024-01-10T02:38:21Z) - Identifiable Latent Polynomial Causal Models Through the Lens of Change [85.67870425656368]
Causal representation learning aims to unveil latent high-level causal representations from observed low-level data.
One of its primary tasks is to provide reliable assurance of identifying these latent causal models, known as identifiability.
arXiv Detail & Related papers (2023-10-24T07:46:10Z) - Theoretical and Practical Perspectives on what Influence Functions Do [45.35457212616306]
Influence functions (IF) have been seen as a technique for explaining model predictions through the lens of the training data.
Recent empirical studies have shown that the existing methods of estimating IF predict the leave-one-out-and-retrain effect poorly.
We show that while most assumptions can be addressed successfully, the parameter divergence poses a clear limitation on the predictive power of IF.
arXiv Detail & Related papers (2023-05-26T14:26:36Z) - Variation-based Cause Effect Identification [5.744133015573047]
We propose a variation-based cause effect identification (VCEI) framework for causal discovery.
Our framework relies on the principle of independence of cause and mechanism (ICM) under the assumption of an existing acyclic causal link.
In the causal direction, such variations are expected to have no impact on the effect generation mechanism.
arXiv Detail & Related papers (2022-11-22T05:19:12Z) - Explainability in Process Outcome Prediction: Guidelines to Obtain
Interpretable and Faithful Models [77.34726150561087]
We define explainability through the interpretability of the explanations and the faithfulness of the explainability model in the field of process outcome prediction.
This paper contributes a set of guidelines named X-MOP which allows selecting the appropriate model based on the event log specifications.
arXiv Detail & Related papers (2022-03-30T05:59:50Z) - Rethinking Stability for Attribution-based Explanations [20.215505482157255]
We introduce metrics to quantify the stability of an explanation and show that several popular explanation methods are unstable.
In particular, we propose new Relative Stability metrics that measure the change in output explanation with respect to change in input, model representation, or output of the underlying predictor.
arXiv Detail & Related papers (2022-03-14T06:19:27Z) - Discovering Latent Causal Variables via Mechanism Sparsity: A New
Principle for Nonlinear ICA [81.4991350761909]
Independent component analysis (ICA) refers to an ensemble of methods which formalize this goal and provide estimation procedure for practical application.
We show that the latent variables can be recovered up to a permutation if one regularizes the latent mechanisms to be sparse.
arXiv Detail & Related papers (2021-07-21T14:22:14Z) - Variational Causal Networks: Approximate Bayesian Inference over Causal
Structures [132.74509389517203]
We introduce a parametric variational family modelled by an autoregressive distribution over the space of discrete DAGs.
In experiments, we demonstrate that the proposed variational posterior is able to provide a good approximation of the true posterior.
arXiv Detail & Related papers (2021-06-14T17:52:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.