Evaluating Model Explanations without Ground Truth

• URL: http://arxiv.org/abs/2505.10399v1
• Date: Thu, 15 May 2025 15:22:06 GMT
• Title: Evaluating Model Explanations without Ground Truth
• Abstract summary: We propose a ground-truth Agnostic eXplanation Evaluation framework (AXE) for evaluating and comparing model explanations.<n>AXE does not require access to ideal ground-truth explanations for comparison, or rely on model sensitivity - providing an independent measure of explanation quality.
• Score: 12.35100095333756
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: There can be many competing and contradictory explanations for a single model prediction, making it difficult to select which one to use. Current explanation evaluation frameworks measure quality by comparing against ideal "ground-truth" explanations, or by verifying model sensitivity to important inputs. We outline the limitations of these approaches, and propose three desirable principles to ground the future development of explanation evaluation strategies for local feature importance explanations. We propose a ground-truth Agnostic eXplanation Evaluation framework (AXE) for evaluating and comparing model explanations that satisfies these principles. Unlike prior approaches, AXE does not require access to ideal ground-truth explanations for comparison, or rely on model sensitivity - providing an independent measure of explanation quality. We verify AXE by comparing with baselines, and show how it can be used to detect explanation fairwashing. Our code is available at https://github.com/KaiRawal/Evaluating-Model-Explanations-without-Ground-Truth.

Related papers

• Evaluating the Ability of Explanations to Disambiguate Models in a Rashomon Set [10.788168368884804]
  We propose three principles of explanation evaluation and a new method "AXE" to evaluate the quality of feature-importance explanations.<n>We show how evaluation metrics that rely on comparing model explanations against ideal ground truth explanations obscure behavioral differences within a Rashomon set.
  arXiv  Detail & Related papers  (2026-01-13T16:31:11Z)
• Estimation of Concept Explanations Should be Uncertainty Aware [39.598213804572396]
  We study a specific kind called Concept Explanations, where the goal is to interpret a model using human-understandable concepts. Although popular for their easy interpretation, concept explanations are known to be noisy. We propose an uncertainty-aware Bayesian estimation method to address these issues, which readily improved the quality of explanations.
  arXiv  Detail & Related papers  (2023-12-13T11:17:27Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• BELLA: Black box model Explanations by Local Linear Approximations [10.71090921516854]
  BELLA is a deterministic model-agnostic post-hoc approach for explaining the individual predictions of regression black-box models.<n>BELLA can produce both factual and counterfactual explanations.
  arXiv  Detail & Related papers  (2023-05-18T21:22:23Z)
• Logical Satisfiability of Counterfactuals for Faithful Explanations in NLI [60.142926537264714]
  We introduce the methodology of Faithfulness-through-Counterfactuals. It generates a counterfactual hypothesis based on the logical predicates expressed in the explanation. It then evaluates if the model's prediction on the counterfactual is consistent with that expressed logic.
  arXiv  Detail & Related papers  (2022-05-25T03:40:59Z)
• Evaluating Local Model-Agnostic Explanations of Learning to Rank Models with Decision Paths [0.0]
  Local explanations of learning-to-rank models are thought to extract the most important features that contribute to the ranking predicted by the model for a single data point. We show that neither of the explanation techniques can achieve an acceptable explanation accuracy when the chosen similarity metric is AUC score or Spearman's rank correlation.
  arXiv  Detail & Related papers  (2022-03-04T13:11:31Z)
• Evaluation of Local Model-Agnostic Explanations Using Ground Truth [4.278336455989584]
  Explanation techniques are commonly evaluated using human-grounded methods. We propose a functionally-grounded evaluation procedure for local model-agnostic explanation techniques.
  arXiv  Detail & Related papers  (2021-06-04T13:47:31Z)
• Search Methods for Sufficient, Socially-Aligned Feature Importance Explanations with In-Distribution Counterfactuals [72.00815192668193]
  Feature importance (FI) estimates are a popular form of explanation, and they are commonly created and evaluated by computing the change in model confidence caused by removing certain input features at test time. We study several under-explored dimensions of FI-based explanations, providing conceptual and empirical improvements for this form of explanation.
  arXiv  Detail & Related papers  (2021-06-01T20:36:48Z)
• To trust or not to trust an explanation: using LEAF to evaluate local linear XAI methods [0.0]
  There is no consensus on how to quantitatively evaluate explanations in practice. explanations are typically used only to inspect black-box models, and the proactive use of explanations as a decision support is generally overlooked. Among the many approaches to XAI, a widely adopted paradigm is Local Linear Explanations - with LIME and SHAP emerging as state-of-the-art methods. We show that these methods are plagued by many defects including unstable explanations, divergence of actual implementations from the promised theoretical properties, and explanations for the wrong label. This highlights the need to have standard and unbiased evaluation procedures for
  arXiv  Detail & Related papers  (2021-06-01T13:14:12Z)
• Evaluating Explanations for Reading Comprehension with Realistic Counterfactuals [26.641834518599303]
  We propose a methodology to evaluate explanations for machine reading comprehension tasks. An explanation should allow us to understand the RC model's high-level behavior with respect to a set of realistic counterfactual input scenarios. Our analysis suggests that pairwise explanation techniques are better suited to RC than token-level attributions.
  arXiv  Detail & Related papers  (2021-04-09T17:55:21Z)
• Contrastive Explanations for Model Interpretability [77.92370750072831]
  We propose a methodology to produce contrastive explanations for classification models. Our method is based on projecting model representation to a latent space. Our findings shed light on the ability of label-contrastive explanations to provide a more accurate and finer-grained interpretability of a model's decision.
  arXiv  Detail & Related papers  (2021-03-02T00:36:45Z)
• Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language? [86.60613602337246]
  We introduce a leakage-adjusted simulatability (LAS) metric for evaluating NL explanations. LAS measures how well explanations help an observer predict a model's output, while controlling for how explanations can directly leak the output. We frame explanation generation as a multi-agent game and optimize explanations for simulatability while penalizing label leakage.
  arXiv  Detail & Related papers  (2020-10-08T16:59:07Z)
• Evaluations and Methods for Explanation through Robustness Analysis [117.7235152610957]
  We establish a novel set of evaluation criteria for such feature based explanations by analysis. We obtain new explanations that are loosely necessary and sufficient for a prediction. We extend the explanation to extract the set of features that would move the current prediction to a target class.
  arXiv  Detail & Related papers  (2020-05-31T05:52:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.