The Meta-Evaluation Problem in Explainable AI: Identifying Reliable
Estimators with MetaQuantus
- URL: http://arxiv.org/abs/2302.07265v2
- Date: Wed, 19 Jul 2023 12:18:34 GMT
- Title: The Meta-Evaluation Problem in Explainable AI: Identifying Reliable
Estimators with MetaQuantus
- Authors: Anna Hedstr\"om, Philine Bommer, Kristoffer K. Wickstr{\o}m, Wojciech
Samek, Sebastian Lapuschkin, Marina M.-C. H\"ohne
- Abstract summary: One of the unsolved challenges in the field of Explainable AI (XAI) is determining how to most reliably estimate the quality of an explanation method.
We address this issue through a meta-evaluation of different quality estimators in XAI.
Our novel framework, MetaQuantus, analyses two complementary performance characteristics of a quality estimator.
- Score: 10.135749005469686
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: One of the unsolved challenges in the field of Explainable AI (XAI) is
determining how to most reliably estimate the quality of an explanation method
in the absence of ground truth explanation labels. Resolving this issue is of
utmost importance as the evaluation outcomes generated by competing evaluation
methods (or ''quality estimators''), which aim at measuring the same property
of an explanation method, frequently present conflicting rankings. Such
disagreements can be challenging for practitioners to interpret, thereby
complicating their ability to select the best-performing explanation method. We
address this problem through a meta-evaluation of different quality estimators
in XAI, which we define as ''the process of evaluating the evaluation method''.
Our novel framework, MetaQuantus, analyses two complementary performance
characteristics of a quality estimator: its resilience to noise and reactivity
to randomness, thus circumventing the need for ground truth labels. We
demonstrate the effectiveness of our framework through a series of experiments,
targeting various open questions in XAI such as the selection and
hyperparameter optimisation of quality estimators. Our work is released under
an open-source license (https://github.com/annahedstroem/MetaQuantus) to serve
as a development tool for XAI- and Machine Learning (ML) practitioners to
verify and benchmark newly constructed quality estimators in a given
explainability context. With this work, we provide the community with clear and
theoretically-grounded guidance for identifying reliable evaluation methods,
thus facilitating reproducibility in the field.
Related papers
- QualEval: Qualitative Evaluation for Model Improvement [82.73561470966658]
We propose QualEval, which augments quantitative scalar metrics with automated qualitative evaluation as a vehicle for model improvement.
QualEval uses a powerful LLM reasoner and our novel flexible linear programming solver to generate human-readable insights.
We demonstrate that leveraging its insights, for example, improves the absolute performance of the Llama 2 model by up to 15% points relative.
arXiv Detail & Related papers (2023-11-06T00:21:44Z) - An Experimental Investigation into the Evaluation of Explainability
Methods [60.54170260771932]
This work compares 14 different metrics when applied to nine state-of-the-art XAI methods and three dummy methods (e.g., random saliency maps) used as references.
Experimental results show which of these metrics produces highly correlated results, indicating potential redundancy.
arXiv Detail & Related papers (2023-05-25T08:07:07Z) - Evaluating Open-QA Evaluation [29.43815593419996]
This study focuses on the evaluation of the Open Question Answering (Open-QA) task, which can directly estimate the factuality of large language models (LLMs)
We introduce a new task, Evaluating QA Evaluation (QA-Eval) and the corresponding dataset EVOUNA, designed to assess the accuracy of AI-generated answers in relation to standard answers within Open-QA.
arXiv Detail & Related papers (2023-05-21T10:40:55Z) - SAFARI: Versatile and Efficient Evaluations for Robustness of
Interpretability [11.230696151134367]
Interpretability of Deep Learning (DL) is a barrier to trustworthy AI.
It is vital to assess how robust DL interpretability is, given an XAI method.
arXiv Detail & Related papers (2022-08-19T16:07:22Z) - Uncertainty-Driven Action Quality Assessment [67.20617610820857]
We propose a novel probabilistic model, named Uncertainty-Driven AQA (UD-AQA), to capture the diversity among multiple judge scores.
We generate the estimation of uncertainty for each prediction, which is employed to re-weight AQA regression loss.
Our proposed method achieves competitive results on three benchmarks including the Olympic events MTL-AQA and FineDiving, and the surgical skill JIGSAWS datasets.
arXiv Detail & Related papers (2022-07-29T07:21:15Z) - From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic
Review on Evaluating Explainable AI [3.7592122147132776]
We identify 12 conceptual properties, such as Compactness and Correctness, that should be evaluated for comprehensively assessing the quality of an explanation.
We find that 1 in 3 papers evaluate exclusively with anecdotal evidence, and 1 in 5 papers evaluate with users.
This systematic collection of evaluation methods provides researchers and practitioners with concrete tools to thoroughly validate, benchmark and compare new and existing XAI methods.
arXiv Detail & Related papers (2022-01-20T13:23:20Z) - GO FIGURE: A Meta Evaluation of Factuality in Summarization [131.1087461486504]
We introduce GO FIGURE, a meta-evaluation framework for evaluating factuality evaluation metrics.
Our benchmark analysis on ten factuality metrics reveals that our framework provides a robust and efficient evaluation.
It also reveals that while QA metrics generally improve over standard metrics that measure factuality across domains, performance is highly dependent on the way in which questions are generated.
arXiv Detail & Related papers (2020-10-24T08:30:20Z) - Uncertainty-aware Score Distribution Learning for Action Quality
Assessment [91.05846506274881]
We propose an uncertainty-aware score distribution learning (USDL) approach for action quality assessment (AQA)
Specifically, we regard an action as an instance associated with a score distribution, which describes the probability of different evaluated scores.
Under the circumstance where fine-grained score labels are available, we devise a multi-path uncertainty-aware score distributions learning (MUSDL) method to explore the disentangled components of a score.
arXiv Detail & Related papers (2020-06-13T15:41:29Z) - Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAI [12.680653816836541]
We propose a ground truth based evaluation framework for XAI methods based on the CLEVR visual question answering task.
Our framework provides a (1) selective, (2) controlled and (3) realistic testbed for the evaluation of neural network explanations.
arXiv Detail & Related papers (2020-03-16T14:43:33Z) - Interpretable Off-Policy Evaluation in Reinforcement Learning by
Highlighting Influential Transitions [48.91284724066349]
Off-policy evaluation in reinforcement learning offers the chance of using observational data to improve future outcomes in domains such as healthcare and education.
Traditional measures such as confidence intervals may be insufficient due to noise, limited data and confounding.
We develop a method that could serve as a hybrid human-AI system, to enable human experts to analyze the validity of policy evaluation estimates.
arXiv Detail & Related papers (2020-02-10T00:26:43Z) - What's a Good Prediction? Challenges in evaluating an agent's knowledge [0.9281671380673306]
We show the conflict between accuracy and usefulness of general knowledge.
We propose an alternate evaluation approach that arises continually in the online continual learning setting.
This paper contributes a first look into evaluation of predictions through their use.
arXiv Detail & Related papers (2020-01-23T21:44:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.