A Unified Study of Machine Learning Explanation Evaluation Metrics

• URL: http://arxiv.org/abs/2203.14265v1
• Date: Sun, 27 Mar 2022 10:12:06 GMT
• Title: A Unified Study of Machine Learning Explanation Evaluation Metrics
• Authors: Yipei Wang, Xiaoqian Wang
• Abstract summary: Many existing metrics for explanations are introduced by researchers as by-products of their proposed explanation techniques to demonstrate the advantages of their methods. We claim that the lack of acknowledged and justified metrics results in chaos in benchmarking these explanation methods. We propose guidelines in dealing with the problems in evaluating machine learning explanation and encourage researchers to carefully deal with these problems when developing explanation techniques and metrics.
• Score: 16.4602888153369
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The growing need for trustworthy machine learning has led to the blossom of interpretability research. Numerous explanation methods have been developed to serve this purpose. However, these methods are deficiently and inappropriately evaluated. Many existing metrics for explanations are introduced by researchers as by-products of their proposed explanation techniques to demonstrate the advantages of their methods. Although widely used, they are more or less accused of problems. We claim that the lack of acknowledged and justified metrics results in chaos in benchmarking these explanation methods -- Do we really have good/bad explanation when a metric gives a high/low score? We split existing metrics into two categories and demonstrate that they are insufficient to properly evaluate explanations for multiple reasons. We propose guidelines in dealing with the problems in evaluating machine learning explanation and encourage researchers to carefully deal with these problems when developing explanation techniques and metrics.

Related papers

• An AI Architecture with the Capability to Explain Recognition Results [0.0]
  This research focuses on the importance of metrics to explainability and contributes two methods yielding performance gains. The first method introduces a combination of explainable and unexplainable flows, proposing a metric to characterize explainability of a decision. The second method compares classic metrics for estimating the effectiveness of neural networks in the system, posing a new metric as the leading performer.
  arXiv  Detail & Related papers  (2024-06-13T02:00:13Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• An Experimental Investigation into the Evaluation of Explainability Methods [60.54170260771932]
  This work compares 14 different metrics when applied to nine state-of-the-art XAI methods and three dummy methods (e.g., random saliency maps) used as references. Experimental results show which of these metrics produces highly correlated results, indicating potential redundancy.
  arXiv  Detail & Related papers  (2023-05-25T08:07:07Z)
• Explanation Selection Using Unlabeled Data for Chain-of-Thought Prompting [80.9896041501715]
  Explanations that have not been "tuned" for a task, such as off-the-shelf explanations written by nonexperts, may lead to mediocre performance. This paper tackles the problem of how to optimize explanation-infused prompts in a blackbox fashion.
  arXiv  Detail & Related papers  (2023-02-09T18:02:34Z)
• Complementary Explanations for Effective In-Context Learning [77.83124315634386]
  Large language models (LLMs) have exhibited remarkable capabilities in learning from explanations in prompts. This work aims to better understand the mechanisms by which explanations are used for in-context learning.
  arXiv  Detail & Related papers  (2022-11-25T04:40:47Z)
• The Solvability of Interpretability Evaluation Metrics [7.3709604810699085]
  Feature attribution methods are often evaluated on metrics such as comprehensiveness and sufficiency. In this paper, we highlight an intriguing property of these metrics: their solvability. We present a series of investigations showing that this beam search explainer is generally comparable or favorable to current choices.
  arXiv  Detail & Related papers  (2022-05-18T02:52:03Z)
• Detection Accuracy for Evaluating Compositional Explanations of Units [5.220940151628734]
  Two examples of methods that use this approach are Network Dissection and Compositional explanations. While intuitively, logical forms are more informative than atomic concepts, it is not clear how to quantify this improvement. We propose to use as evaluation metric the Detection Accuracy, which measures units' consistency of detection of their assigned explanations.
  arXiv  Detail & Related papers  (2021-09-16T08:47:34Z)
• Towards Benchmarking the Utility of Explanations for Model Debugging [13.135013586592585]
  We argue the need for a benchmark to facilitate evaluations of the utility of post-hoc explanation methods. We highlight that such a benchmark facilitates not only assessing the effectiveness of explanations but also their efficiency.
  arXiv  Detail & Related papers  (2021-05-10T16:57:33Z)
• Evaluating Explanations: How much do explanations from the teacher aid students? [103.05037537415811]
  We formalize the value of explanations using a student-teacher paradigm that measures the extent to which explanations improve student models in learning. Unlike many prior proposals to evaluate explanations, our approach cannot be easily gamed, enabling principled, scalable, and automatic evaluation of attributions.
  arXiv  Detail & Related papers  (2020-12-01T23:40:21Z)
• Evaluation of Similarity-based Explanations [36.10585276728203]
  We investigated relevance metrics that can provide reasonable explanations to users. Our experiments revealed that the cosine similarity of the gradients of the loss performs best. Some metrics perform poorly in our tests and analyzed the reasons of their failure.
  arXiv  Detail & Related papers  (2020-06-08T12:39:46Z)
• Evaluating Explanation Methods for Neural Machine Translation [44.836653992441455]
  We propose a principled metric based on fidelity in regard to the predictive behavior of the NMT model. On six standard translation tasks, we quantitatively evaluate several explanation methods in terms of the proposed metric.
  arXiv  Detail & Related papers  (2020-05-04T17:26:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.