BASED-XAI: Breaking Ablation Studies Down for Explainable Artificial
Intelligence
- URL: http://arxiv.org/abs/2207.05566v1
- Date: Tue, 12 Jul 2022 14:38:37 GMT
- Title: BASED-XAI: Breaking Ablation Studies Down for Explainable Artificial
Intelligence
- Authors: Isha Hameed, Samuel Sharpe, Daniel Barcklow, Justin Au-Yeung, Sahil
Verma, Jocelyn Huang, Brian Barr, C. Bayan Bruss
- Abstract summary: We show how varying perturbations can help to avoid potentially flawed conclusions.
We also show how treatment of categorical variables is an important consideration in both post-hoc explainability and ablation studies.
- Score: 1.2948254191169823
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Explainable artificial intelligence (XAI) methods lack ground truth. In its
place, method developers have relied on axioms to determine desirable
properties for their explanations' behavior. For high stakes uses of machine
learning that require explainability, it is not sufficient to rely on axioms as
the implementation, or its usage, can fail to live up to the ideal. As a
result, there exists active research on validating the performance of XAI
methods. The need for validation is especially magnified in domains with a
reliance on XAI. A procedure frequently used to assess their utility, and to
some extent their fidelity, is an ablation study. By perturbing the input
variables in rank order of importance, the goal is to assess the sensitivity of
the model's performance. Perturbing important variables should correlate with
larger decreases in measures of model capability than perturbing less important
features. While the intent is clear, the actual implementation details have not
been studied rigorously for tabular data. Using five datasets, three XAI
methods, four baselines, and three perturbations, we aim to show 1) how varying
perturbations and adding simple guardrails can help to avoid potentially flawed
conclusions, 2) how treatment of categorical variables is an important
consideration in both post-hoc explainability and ablation studies, and 3) how
to identify useful baselines for XAI methods and viable perturbations for
ablation studies.
Related papers
- XAI-TRIS: Non-linear image benchmarks to quantify false positive
post-hoc attribution of feature importance [1.3958169829527285]
A lack of formal underpinning leaves it unclear as to what conclusions can safely be drawn from the results of a given XAI method.
This means that challenging non-linear problems, typically solved by deep neural networks, presently lack appropriate remedies.
We show that popular XAI methods are often unable to significantly outperform random performance baselines and edge detection methods.
arXiv Detail & Related papers (2023-06-22T11:31:11Z) - Explaining Explainability: Towards Deeper Actionable Insights into Deep
Learning through Second-order Explainability [70.60433013657693]
Second-order explainable AI (SOXAI) was recently proposed to extend explainable AI (XAI) from the instance level to the dataset level.
We demonstrate for the first time, via example classification and segmentation cases, that eliminating irrelevant concepts from the training set based on actionable insights from SOXAI can enhance a model's performance.
arXiv Detail & Related papers (2023-06-14T23:24:01Z) - Interpretability at Scale: Identifying Causal Mechanisms in Alpaca [62.65877150123775]
We use Boundless DAS to efficiently search for interpretable causal structure in large language models while they follow instructions.
Our findings mark a first step toward faithfully understanding the inner-workings of our ever-growing and most widely deployed language models.
arXiv Detail & Related papers (2023-05-15T17:15:40Z) - Trying to Outrun Causality with Machine Learning: Limitations of Model
Explainability Techniques for Identifying Predictive Variables [7.106986689736828]
We show that machine learning algorithms are not as flexible as they might seem, and are instead incredibly sensitive to the underling causal structure in the data.
We provide some alternative recommendations for researchers wanting to explore the data for important variables.
arXiv Detail & Related papers (2022-02-20T17:48:54Z) - Scrutinizing XAI using linear ground-truth data with suppressor
variables [0.8602553195689513]
Saliency methods rank input features according to some measure of 'importance'
It has been demonstrated that some saliency methods can highlight features that have no statistical association with the prediction target (suppressor variables)
arXiv Detail & Related papers (2021-11-14T23:02:02Z) - Logic Constraints to Feature Importances [17.234442722611803]
"Black box" nature of AI models is often a limit for a reliable application in high-stakes fields like diagnostic techniques, autonomous guide, etc.
Recent works have shown that an adequate level of interpretability could enforce the more general concept of model trustworthiness.
The basic idea of this paper is to exploit the human prior knowledge of the features' importance for a specific task, in order to coherently aid the phase of the model's fitting.
arXiv Detail & Related papers (2021-10-13T09:28:38Z) - Systematic Evaluation of Causal Discovery in Visual Model Based
Reinforcement Learning [76.00395335702572]
A central goal for AI and causality is the joint discovery of abstract representations and causal structure.
Existing environments for studying causal induction are poorly suited for this objective because they have complicated task-specific causal graphs.
In this work, our goal is to facilitate research in learning representations of high-level variables as well as causal structures among them.
arXiv Detail & Related papers (2021-07-02T05:44:56Z) - Counterfactual Explanations as Interventions in Latent Space [62.997667081978825]
Counterfactual explanations aim to provide to end users a set of features that need to be changed in order to achieve a desired outcome.
Current approaches rarely take into account the feasibility of actions needed to achieve the proposed explanations.
We present Counterfactual Explanations as Interventions in Latent Space (CEILS), a methodology to generate counterfactual explanations.
arXiv Detail & Related papers (2021-06-14T20:48:48Z) - Data Representing Ground-Truth Explanations to Evaluate XAI Methods [0.0]
Explainable artificial intelligence (XAI) methods are currently evaluated with approaches mostly originated in interpretable machine learning (IML) research.
We propose to represent explanations with canonical equations that can be used to evaluate the accuracy of XAI methods.
arXiv Detail & Related papers (2020-11-18T16:54:53Z) - Evaluating Explainable AI: Which Algorithmic Explanations Help Users
Predict Model Behavior? [97.77183117452235]
We carry out human subject tests to isolate the effect of algorithmic explanations on model interpretability.
Clear evidence of method effectiveness is found in very few cases.
Our results provide the first reliable and comprehensive estimates of how explanations influence simulatability.
arXiv Detail & Related papers (2020-05-04T20:35:17Z) - Fairness by Learning Orthogonal Disentangled Representations [50.82638766862974]
We propose a novel disentanglement approach to invariant representation problem.
We enforce the meaningful representation to be agnostic to sensitive information by entropy.
The proposed approach is evaluated on five publicly available datasets.
arXiv Detail & Related papers (2020-03-12T11:09:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.