Evaluating Explainability: A Framework for Systematic Assessment and Reporting of Explainable AI Features

• URL: http://arxiv.org/abs/2506.13917v1
• Date: Mon, 16 Jun 2025 18:51:46 GMT
• Title: Evaluating Explainability: A Framework for Systematic Assessment and Reporting of Explainable AI Features
• Authors: Miguel A. Lago, Ghada Zamzmi, Brandon Eich, Jana G. Delfino,
• Abstract summary: We propose a framework to assess and report explainable AI features.<n>Our evaluation framework is based on four criteria: 1) Consistency quantifies the variability of explanations to similar inputs, 2) Plausibility estimates how close the explanation is to the ground truth, 3) Fidelity assesses the alignment between the explanation and the model internal mechanisms, and 4) Usefulness evaluates the impact on task performance.
• Score: 2.4458403938995064
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Explainability features are intended to provide insight into the internal mechanisms of an AI device, but there is a lack of evaluation techniques for assessing the quality of provided explanations. We propose a framework to assess and report explainable AI features. Our evaluation framework for AI explainability is based on four criteria: 1) Consistency quantifies the variability of explanations to similar inputs, 2) Plausibility estimates how close the explanation is to the ground truth, 3) Fidelity assesses the alignment between the explanation and the model internal mechanisms, and 4) Usefulness evaluates the impact on task performance of the explanation. Finally, we developed a scorecard for AI explainability methods that serves as a complete description and evaluation to accompany this type of algorithm. We describe these four criteria and give examples on how they can be evaluated. As a case study, we use Ablation CAM and Eigen CAM to illustrate the evaluation of explanation heatmaps on the detection of breast lesions on synthetic mammographies. The first three criteria are evaluated for clinically-relevant scenarios. Our proposed framework establishes criteria through which the quality of explanations provided by AI models can be evaluated. We intend for our framework to spark a dialogue regarding the value provided by explainability features and help improve the development and evaluation of AI-based medical devices.

Related papers

• A Systematic Review of User-Centred Evaluation of Explainable AI in Healthcare [1.57531613028502]
  This study aims to develop a framework of well-defined, atomic properties that characterise the user experience of XAI in healthcare.<n>We also provide context-sensitive guidelines for defining evaluation strategies based on system characteristics.
  arXiv  Detail & Related papers  (2025-06-16T18:30:00Z)
• The Explanation Necessity for Healthcare AI [3.8953842074141387]
  We propose a novel categorization system comprising four classes of explanation necessity.<n>This framework provides a practical tool for researchers to determine the appropriate depth of explainability needed.
  arXiv  Detail & Related papers  (2024-05-31T22:20:10Z)
• Introducing User Feedback-based Counterfactual Explanations (UFCE) [49.1574468325115]
  Counterfactual explanations (CEs) have emerged as a viable solution for generating comprehensible explanations in XAI. UFCE allows for the inclusion of user constraints to determine the smallest modifications in the subset of actionable features. UFCE outperforms two well-known CE methods in terms of textitproximity, textitsparsity, and textitfeasibility.
  arXiv  Detail & Related papers  (2024-02-26T20:09:44Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• Towards a Comprehensive Human-Centred Evaluation Framework for Explainable AI [1.7222662622390634]
  We propose to adapt the User-Centric Evaluation Framework used in recommender systems. We integrate explanation aspects, summarise explanation properties, indicate relations between them, and categorise metrics that measure these properties.
  arXiv  Detail & Related papers  (2023-07-31T09:20:16Z)
• Informing clinical assessment by contextualizing post-hoc explanations of risk prediction models in type-2 diabetes [50.8044927215346]
  We consider a comorbidity risk prediction scenario and focus on contexts regarding the patients clinical state. We employ several state-of-the-art LLMs to present contexts around risk prediction model inferences and evaluate their acceptability. Our paper is one of the first end-to-end analyses identifying the feasibility and benefits of contextual explanations in a real-world clinical use case.
  arXiv  Detail & Related papers  (2023-02-11T18:07:11Z)
• REV: Information-Theoretic Evaluation of Free-Text Rationales [83.24985872655738]
  We argue that an ideal metric should focus on the new information uniquely provided in the rationale that is otherwise not provided in the input or the label. We propose a metric called REV (Rationale Evaluation with conditional V-information) to quantify the amount of new, label-relevant information in a rationale.
  arXiv  Detail & Related papers  (2022-10-10T19:31:30Z)
• Neural Causal Models for Counterfactual Identification and Estimation [62.30444687707919]
  We study the evaluation of counterfactual statements through neural models. First, we show that neural causal models (NCMs) are expressive enough. Second, we develop an algorithm for simultaneously identifying and estimating counterfactual distributions.
  arXiv  Detail & Related papers  (2022-09-30T18:29:09Z)
• A Meta Survey of Quality Evaluation Criteria in Explanation Methods [0.5801044612920815]
  Explanation methods and their evaluation have become a significant issue in explainable artificial intelligence (XAI) Since the most accurate AI models are opaque with low transparency and comprehensibility, explanations are essential for bias detection and control of uncertainty. There are a plethora of criteria to choose from when evaluating explanation method quality.
  arXiv  Detail & Related papers  (2022-03-25T22:24:21Z)
• A Diagnostic Study of Explainability Techniques for Text Classification [52.879658637466605]
  We develop a list of diagnostic properties for evaluating existing explainability techniques. We compare the saliency scores assigned by the explainability techniques with human annotations of salient input regions to find relations between a model's performance and the agreement of its rationales with human ones.
  arXiv  Detail & Related papers  (2020-09-25T12:01:53Z)
• The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies [1.2762298148425795]
  Lack of transparency is identified as one of the main barriers to implementation of AI systems in health care. We review the recent literature to provide guidance to researchers and practitioners on the design of explainable AI systems. We conclude that explainable modelling can contribute to trustworthy AI, but the benefits of explainability still need to be proven in practice.
  arXiv  Detail & Related papers  (2020-07-31T09:08:27Z)
• Evaluations and Methods for Explanation through Robustness Analysis [117.7235152610957]
  We establish a novel set of evaluation criteria for such feature based explanations by analysis. We obtain new explanations that are loosely necessary and sufficient for a prediction. We extend the explanation to extract the set of features that would move the current prediction to a target class.
  arXiv  Detail & Related papers  (2020-05-31T05:52:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.