Data Representing Ground-Truth Explanations to Evaluate XAI Methods
- URL: http://arxiv.org/abs/2011.09892v1
- Date: Wed, 18 Nov 2020 16:54:53 GMT
- Title: Data Representing Ground-Truth Explanations to Evaluate XAI Methods
- Authors: Shideh Shams Amiri, Rosina O. Weber, Prateek Goel, Owen Brooks, Archer
Gandley, Brian Kitchell, Aaron Zehm
- Abstract summary: Explainable artificial intelligence (XAI) methods are currently evaluated with approaches mostly originated in interpretable machine learning (IML) research.
We propose to represent explanations with canonical equations that can be used to evaluate the accuracy of XAI methods.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Explainable artificial intelligence (XAI) methods are currently evaluated
with approaches mostly originated in interpretable machine learning (IML)
research that focus on understanding models such as comparison against existing
attribution approaches, sensitivity analyses, gold set of features, axioms, or
through demonstration of images. There are problems with these methods such as
that they do not indicate where current XAI approaches fail to guide
investigations towards consistent progress of the field. They do not measure
accuracy in support of accountable decisions, and it is practically impossible
to determine whether one XAI method is better than the other or what the
weaknesses of existing models are, leaving researchers without guidance on
which research questions will advance the field. Other fields usually utilize
ground-truth data and create benchmarks. Data representing ground-truth
explanations is not typically used in XAI or IML. One reason is that
explanations are subjective, in the sense that an explanation that satisfies
one user may not satisfy another. To overcome these problems, we propose to
represent explanations with canonical equations that can be used to evaluate
the accuracy of XAI methods. The contributions of this paper include a
methodology to create synthetic data representing ground-truth explanations,
three data sets, an evaluation of LIME using these data sets, and a preliminary
analysis of the challenges and potential benefits in using these data to
evaluate existing XAI approaches. Evaluation methods based on human-centric
studies are outside the scope of this paper.
Related papers
- Explainable AI needs formal notions of explanation correctness [2.1309989863595677]
Machine learning in critical domains such as medicine poses risks and requires regulation.
One requirement is that decisions of ML systems in high-risk applications should be human-understandable.
In its current form, XAI is unfit to provide quality control for ML; it itself needs scrutiny.
arXiv Detail & Related papers (2024-09-22T20:47:04Z) - Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development.
To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps.
These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
arXiv Detail & Related papers (2023-12-10T23:13:23Z) - Assessing Fidelity in XAI post-hoc techniques: A Comparative Study with
Ground Truth Explanations Datasets [0.0]
XAI methods based on the backpropagation of output information to input yield higher accuracy and reliability.
Backpropagation method tends to generate more noisy saliency maps.
Findings have significant implications for the advancement of XAI methods.
arXiv Detail & Related papers (2023-11-03T14:57:24Z) - What's meant by explainable model: A Scoping Review [0.38252451346419336]
This paper investigates whether the term explainable model is adopted by authors under the assumption that incorporating a post-hoc XAI method suffices to characterize a model as explainable.
We found that 81% of the application papers that refer to their approaches as an explainable model do not conduct any form of evaluation on the XAI method they used.
arXiv Detail & Related papers (2023-07-18T22:55:04Z) - XAI-TRIS: Non-linear image benchmarks to quantify false positive
post-hoc attribution of feature importance [1.3958169829527285]
A lack of formal underpinning leaves it unclear as to what conclusions can safely be drawn from the results of a given XAI method.
This means that challenging non-linear problems, typically solved by deep neural networks, presently lack appropriate remedies.
We show that popular XAI methods are often unable to significantly outperform random performance baselines and edge detection methods.
arXiv Detail & Related papers (2023-06-22T11:31:11Z) - Explaining Explainability: Towards Deeper Actionable Insights into Deep
Learning through Second-order Explainability [70.60433013657693]
Second-order explainable AI (SOXAI) was recently proposed to extend explainable AI (XAI) from the instance level to the dataset level.
We demonstrate for the first time, via example classification and segmentation cases, that eliminating irrelevant concepts from the training set based on actionable insights from SOXAI can enhance a model's performance.
arXiv Detail & Related papers (2023-06-14T23:24:01Z) - An Experimental Investigation into the Evaluation of Explainability
Methods [60.54170260771932]
This work compares 14 different metrics when applied to nine state-of-the-art XAI methods and three dummy methods (e.g., random saliency maps) used as references.
Experimental results show which of these metrics produces highly correlated results, indicating potential redundancy.
arXiv Detail & Related papers (2023-05-25T08:07:07Z) - Connecting Algorithmic Research and Usage Contexts: A Perspective of
Contextualized Evaluation for Explainable AI [65.44737844681256]
A lack of consensus on how to evaluate explainable AI (XAI) hinders the advancement of the field.
We argue that one way to close the gap is to develop evaluation methods that account for different user requirements.
arXiv Detail & Related papers (2022-06-22T05:17:33Z) - Beyond Explaining: Opportunities and Challenges of XAI-Based Model
Improvement [75.00655434905417]
Explainable Artificial Intelligence (XAI) is an emerging research field bringing transparency to highly complex machine learning (ML) models.
This paper offers a comprehensive overview over techniques that apply XAI practically for improving various properties of ML models.
We show empirically through experiments on toy and realistic settings how explanations can help improve properties such as model generalization ability or reasoning.
arXiv Detail & Related papers (2022-03-15T15:44:28Z) - Human-in-the-Loop Disinformation Detection: Stance, Sentiment, or
Something Else? [93.91375268580806]
Both politics and pandemics have recently provided ample motivation for the development of machine learning-enabled disinformation (a.k.a. fake news) detection algorithms.
Existing literature has focused primarily on the fully-automated case, but the resulting techniques cannot reliably detect disinformation on the varied topics, sources, and time scales required for military applications.
By leveraging an already-available analyst as a human-in-the-loop, canonical machine learning techniques of sentiment analysis, aspect-based sentiment analysis, and stance detection become plausible methods to use for a partially-automated disinformation detection system.
arXiv Detail & Related papers (2021-11-09T13:30:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.