Metamorphic Testing with the Rashomon Set: Explanation Faithfulness in Machine Learning

• URL: http://arxiv.org/abs/2606.06056v1
• Date: Thu, 04 Jun 2026 11:57:26 GMT
• Title: Metamorphic Testing with the Rashomon Set: Explanation Faithfulness in Machine Learning
• Abstract summary: Multiple machine learning models can achieve near-equivalent predictive performance on the same task, yet provide divergent feature-based explanations.<n>This is called the Rashomon effect of (explainable) machine learning, and it raises the question of which explanations, if any, are trustworthy.<n>We propose a framework based on metamorphic testing that assesses explanation faithfulness without requiring ground-truth labels.
• Score: 4.570003973862485
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multiple machine learning models can achieve near-equivalent predictive performance on the same task, yet provide divergent feature-based explanations. This is called the Rashomon effect of (explainable) machine learning, and it raises the question of which explanations, if any, are trustworthy. We propose a framework based on metamorphic testing that assesses explanation faithfulness without requiring ground-truth labels by exploring attributed feature importance from post-hoc explanation methods. Five metamorphic relations formalize expected consistency properties between model behavior and feature attributions. We apply this general framework to two tabular regression datasets and two post-hoc explainers (SHAP and LIME) to demonstrate the approach. The framework offers a practical, model-agnostic tool for selecting accurate models with reliable and trustworthy explanations.

Related papers

• Evaluating Local Explainability Metrics for Machine Learning Models on Tabular Data [1.423280626666929]
  An explanation can appear plausible to humans but fail to capture the internal reasoning of a model.<n>This paper studies the trustworthiness of local explainability techniques when applied to complex tabular classification tasks.
  arXiv  Detail & Related papers  (2026-05-26T19:38:10Z)
• Hypothesis Class Determines Explanation: Why Accurate Models Disagree on Feature Attribution [0.0]
  We find that models with identical predictive behavior can produce substantially different feature attributions.<n>We identify hypothesis class as the structural driver of this phenomenon, which we term the Explanation Lottery.<n>Our results demonstrate that model selection is not explanation-neutral: the hypothesis class chosen for deployment can determine which features are attributed responsibility for a decision.
  arXiv  Detail & Related papers  (2026-03-16T18:55:50Z)
• Why Does It Look There? Structured Explanations for Image Classification [31.62439146845817]
  Interpretability to Explainability (I2X) is a framework that builds structured explanations directly from unstructured interpretability.<n>We show I2X can be used to improve predictions across different model architectures and datasets.
  arXiv  Detail & Related papers  (2026-03-10T21:15:14Z)
• Does the Model Say What the Data Says? A Simple Heuristic for Model Data Alignment [0.0]
  We propose a framework to evaluate whether machine learning models align with the structure of the data they learn from.<n>Unlike existing interpretability methods that focus exclusively on explaining model behavior, our approach establishes a baseline derived directly from the data itself.
  arXiv  Detail & Related papers  (2025-11-26T21:44:55Z)
• Explainability for Machine Learning Models: From Data Adaptability to User Perception [0.8702432681310401]
  This thesis explores the generation of local explanations for already deployed machine learning models. It aims to identify optimal conditions for producing meaningful explanations considering both data and user requirements.
  arXiv  Detail & Related papers  (2024-02-16T18:44:37Z)
• Discriminative Feature Attributions: Bridging Post Hoc Explainability and Inherent Interpretability [29.459228981179674]
  Post hoc explanations incorrectly attribute high importance to features that are unimportant or non-discriminative for the underlying task. Inherently interpretable models, on the other hand, circumvent these issues by explicitly encoding explanations into model architecture. We propose Distractor Erasure Tuning (DiET), a method that adapts black-box models to be robust to distractor erasure.
  arXiv  Detail & Related papers  (2023-07-27T17:06:02Z)
• On the Joint Interaction of Models, Data, and Features [82.60073661644435]
  We introduce a new tool, the interaction tensor, for empirically analyzing the interaction between data and model through features. Based on these observations, we propose a conceptual framework for feature learning. Under this framework, the expected accuracy for a single hypothesis and agreement for a pair of hypotheses can both be derived in closed-form.
  arXiv  Detail & Related papers  (2023-06-07T21:35:26Z)
• Learning with Explanation Constraints [91.23736536228485]
  We provide a learning theoretic framework to analyze how explanations can improve the learning of our models. We demonstrate the benefits of our approach over a large array of synthetic and real-world experiments.
  arXiv  Detail & Related papers  (2023-03-25T15:06:47Z)
• Explainability in Process Outcome Prediction: Guidelines to Obtain Interpretable and Faithful Models [77.34726150561087]
  We define explainability through the interpretability of the explanations and the faithfulness of the explainability model in the field of process outcome prediction. This paper contributes a set of guidelines named X-MOP which allows selecting the appropriate model based on the event log specifications.
  arXiv  Detail & Related papers  (2022-03-30T05:59:50Z)
• Partial Order in Chaos: Consensus on Feature Attributions in the Rashomon Set [50.67431815647126]
  Post-hoc global/local feature attribution methods are being progressively employed to understand machine learning models. We show that partial orders of local/global feature importance arise from this methodology. We show that every relation among features present in these partial orders also holds in the rankings provided by existing approaches.
  arXiv  Detail & Related papers  (2021-10-26T02:53:14Z)
• Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanations [64.85696493596821]
  In computer vision applications, generative counterfactual methods indicate how to perturb a model's input to change its prediction. We propose a counterfactual method that learns a perturbation in a disentangled latent space that is constrained using a diversity-enforcing loss. Our model improves the success rate of producing high-quality valuable explanations when compared to previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-03-18T12:57:34Z)
• Deducing neighborhoods of classes from a fitted model [68.8204255655161]
  In this article a new kind of interpretable machine learning method is presented. It can help to understand the partitioning of the feature space into predicted classes in a classification model using quantile shifts. Basically, real data points (or specific points of interest) are used and the changes of the prediction after slightly raising or decreasing specific features are observed.
  arXiv  Detail & Related papers  (2020-09-11T16:35:53Z)
• Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior? [97.77183117452235]
  We carry out human subject tests to isolate the effect of algorithmic explanations on model interpretability. Clear evidence of method effectiveness is found in very few cases. Our results provide the first reliable and comprehensive estimates of how explanations influence simulatability.
  arXiv  Detail & Related papers  (2020-05-04T20:35:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.