The Blame Problem in Evaluating Local Explanations, and How to Tackle it
- URL: http://arxiv.org/abs/2310.03466v1
- Date: Thu, 5 Oct 2023 11:21:49 GMT
- Title: The Blame Problem in Evaluating Local Explanations, and How to Tackle it
- Authors: Amir Hossein Akhavan Rahnama
- Abstract summary: Bar for developing new explainability techniques is low due to lack of optimal evaluation measures.
Without rigorous measures, it is hard to have concrete evidence of whether new explanation techniques can significantly outperform their predecessors.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The number of local model-agnostic explanation techniques proposed has grown
rapidly recently. One main reason is that the bar for developing new
explainability techniques is low due to the lack of optimal evaluation
measures. Without rigorous measures, it is hard to have concrete evidence of
whether the new explanation techniques can significantly outperform their
predecessors. Our study proposes a new taxonomy for evaluating local
explanations: robustness, evaluation using ground truth from synthetic datasets
and interpretable models, model randomization, and human-grounded evaluation.
Using this proposed taxonomy, we highlight that all categories of evaluation
methods, except those based on the ground truth from interpretable models,
suffer from a problem we call the "blame problem." In our study, we argue that
this category of evaluation measure is a more reasonable method for evaluating
local model-agnostic explanations. However, we show that even this category of
evaluation measures has further limitations. The evaluation of local
explanations remains an open research problem.
Related papers
- Pitfalls of topology-aware image segmentation [81.19923502845441]
We identify critical pitfalls in model evaluation that include inadequate connectivity choices, overlooked topological artifacts, and inappropriate use of evaluation metrics.
We propose a set of actionable recommendations to establish fair and robust evaluation standards for topology-aware medical image segmentation methods.
arXiv Detail & Related papers (2024-12-19T08:11:42Z) - Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development.
To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps.
These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
arXiv Detail & Related papers (2023-12-10T23:13:23Z) - On The Coherence of Quantitative Evaluation of Visual Explanations [0.7212939068975619]
Evaluation methods have been proposed to assess the "goodness" of visual explanations.
We study a subset of the ImageNet-1k validation set where we evaluate a number of different commonly-used explanation methods.
Results of our study suggest that there is a lack of coherency on the grading provided by some of the considered evaluation methods.
arXiv Detail & Related papers (2023-02-14T13:41:57Z) - Are Neural Topic Models Broken? [81.15470302729638]
We study the relationship between automated and human evaluation of topic models.
We find that neural topic models fare worse in both respects compared to an established classical method.
arXiv Detail & Related papers (2022-10-28T14:38:50Z) - Off-policy evaluation for learning-to-rank via interpolating the
item-position model and the position-based model [83.83064559894989]
A critical need for industrial recommender systems is the ability to evaluate recommendation policies offline, before deploying them to production.
We develop a new estimator that mitigates the problems of the two most popular off-policy estimators for rankings.
In particular, the new estimator, called INTERPOL, addresses the bias of a potentially misspecified position-based model.
arXiv Detail & Related papers (2022-10-15T17:22:30Z) - CEBaB: Estimating the Causal Effects of Real-World Concepts on NLP Model
Behavior [26.248879735549277]
We cast model explanation as the causal inference problem of estimating causal effects of real-world concepts on the output behavior of ML models.
We introduce CEBaB, a new benchmark dataset for assessing concept-based explanation methods in Natural Language Processing (NLP)
We use CEBaB to compare the quality of a range of concept-based explanation methods covering different assumptions and conceptions of the problem.
arXiv Detail & Related papers (2022-05-27T17:59:14Z) - Explain, Edit, and Understand: Rethinking User Study Design for
Evaluating Model Explanations [97.91630330328815]
We conduct a crowdsourcing study, where participants interact with deception detection models that have been trained to distinguish between genuine and fake hotel reviews.
We observe that for a linear bag-of-words model, participants with access to the feature coefficients during training are able to cause a larger reduction in model confidence in the testing phase when compared to the no-explanation control.
arXiv Detail & Related papers (2021-12-17T18:29:56Z) - When Stability meets Sufficiency: Informative Explanations that do not Overwhelm [15.897648942908747]
We consider features-based attribution methods that highlight what should be minimally sufficient to justify the classification of an input.
While minimal sufficiency is an attractive property akin to comprehensibility, the resulting explanations are often too sparse for a human to understand and evaluate the local behavior of the model.
We propose a novel method called Path-Sufficient Explanations Method (PSEM) that outputs a sequence of stable and sufficient explanations for a given input.
arXiv Detail & Related papers (2021-09-13T16:06:10Z) - Evaluation of Local Model-Agnostic Explanations Using Ground Truth [4.278336455989584]
Explanation techniques are commonly evaluated using human-grounded methods.
We propose a functionally-grounded evaluation procedure for local model-agnostic explanation techniques.
arXiv Detail & Related papers (2021-06-04T13:47:31Z) - Evaluations and Methods for Explanation through Robustness Analysis [117.7235152610957]
We establish a novel set of evaluation criteria for such feature based explanations by analysis.
We obtain new explanations that are loosely necessary and sufficient for a prediction.
We extend the explanation to extract the set of features that would move the current prediction to a target class.
arXiv Detail & Related papers (2020-05-31T05:52:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.