Rethinking XAI Evaluation: A Human-Centered Audit of Shapley Benchmarks in High-Stakes Settings

• URL: http://arxiv.org/abs/2604.22662v1
• Date: Fri, 24 Apr 2026 15:38:44 GMT
• Title: Rethinking XAI Evaluation: A Human-Centered Audit of Shapley Benchmarks in High-Stakes Settings
• Abstract summary: Shapley values are a cornerstone of explainable AI, yet their proliferation into competing formulations has created a fragmented landscape.<n>We use a unified amortized framework to isolate semantic differences between eight Shapley variants.<n>Our results reveal a fundamental misalignment: standard quantitative metrics, such as sparsity and faithfulness, are decoupled from human-perceived clarity and decision utility.
• Score: 8.934085986179019
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Shapley values are a cornerstone of explainable AI, yet their proliferation into competing formulations has created a fragmented landscape with little consensus on practical deployment. While theoretical differences are well-documented, evaluation remains reliant on quantitative proxies whose alignment with human utility is unverified. In this work, we use a unified amortized framework to isolate semantic differences between eight Shapley variants under the low-latency constraints of operational risk workflows. We conduct a large-scale empirical evaluation across four risk datasets and a realistic fraud-detection environment involving professional analysts and 3,735 case reviews. Our results reveal a fundamental misalignment: standard quantitative metrics, such as sparsity and faithfulness, are decoupled from human-perceived clarity and decision utility. Furthermore, while no formulation improved objective analyst performance, explanations consistently increased decision confidence, signaling a critical risk of automation bias in high-stakes settings. These findings suggest that current evaluation proxies are insufficient for predicting downstream human impact, and we provide evidence-based guidance for selecting formulations and metrics in operational decision systems.

Related papers

• Human-Anchored Factuality Evaluation with Strategic Annotation [5.771467270404003]
  LLM-based factuality judges provide scalable evaluation signals, but their metrics are often systematically biased relative to human judgments.<n>We study human-anchored factuality evaluation under limited annotation budgets, where judge predictions on the full dataset are combined with human labels on a small selectively sampled subset to obtain statistically valid estimates.<n>Our FSA-guided policy improves annotation efficiency over uniform sampling and uncertainty-driven baselines, achieving effective-sample-size gains of 40.3% on AutoFA and 27.1% on RAGTruth.
  arXiv  Detail & Related papers  (2026-08-31T23:50:10Z)
• A Full-Pipeline Framework for Evaluating Membership Inference Attacks in Machine Learning [31.096361601691708]
  We introduce a comprehensive evaluation framework that characterizes privacy risks across the entire machine learning pipeline.<n>Our framework rigorously evaluates state-of-the-art MIAs across a broad spectrum of training configurations.
  arXiv  Detail & Related papers  (2026-05-28T06:48:22Z)
• Aligning LLM Uncertainty with Human Disagreement in Subjectivity Analysis [57.78407973423517]
  We advocate uncertainty-aware subjectivity analysis, where models are expected to make predictions while expressing uncertainty that reflects human disagreement.<n>To operationalize this perspective, we propose a two-phase Disagreement Perception and Uncertainty Alignment framework.<n> Experiments on three subjectivity analysis tasks show that DPUA preserves task performance while better aligning model uncertainty with human disagreement.
  arXiv  Detail & Related papers  (2026-05-11T11:52:58Z)
• Beyond Benchmark Islands: Toward Representative Trustworthiness Evaluation for Agentic AI [45.21562889170875]
  We propose a systematic evaluation paradigm that characterizes agent trustworthiness over a scenario manifold.<n>The framework integrates four complementary components: static cognitive and policy analysis, (ii) interactive sandbox simulation, (iii) social-ethical alignment assessment, and (iv) a distribution-aware representative sampling engine.
  arXiv  Detail & Related papers  (2026-03-16T08:51:33Z)
• Toward Quantitative Modeling of Cybersecurity Risks Due to AI Misuse [50.87630846876635]
  We develop nine detailed cyber risk models.<n>Each model decomposes attacks into steps using the MITRE ATT&CK framework.<n>Individual estimates are aggregated through Monte Carlo simulation.
  arXiv  Detail & Related papers  (2025-12-09T17:54:17Z)
• Addressing Pitfalls in the Evaluation of Uncertainty Estimation Methods for Natural Language Generation [20.726685669562496]
  Hallucinations are a common issue that undermine the reliability of large language models (LLMs)<n>Recent studies have identified a subset of hallucinations, known as confabulations, which arise due to predictive uncertainty of LLMs.<n>To detect confabulations, various methods for estimating predictive uncertainty in natural language generation (NLG) have been developed.
  arXiv  Detail & Related papers  (2025-10-02T17:54:09Z)
• RADAR: A Risk-Aware Dynamic Multi-Agent Framework for LLM Safety Evaluation via Role-Specialized Collaboration [81.38705556267917]
  Existing safety evaluation methods for large language models (LLMs) suffer from inherent limitations.<n>We introduce a theoretical framework that reconstructs the underlying risk concept space.<n>We propose RADAR, a multi-agent collaborative evaluation framework.
  arXiv  Detail & Related papers  (2025-09-28T09:35:32Z)
• Conformalized Decision Risk Assessment [5.391713612899277]
  We introduce CREDO, a novel framework that quantifies for any candidate decision, a distribution-free upper bound on the probability that the decision is suboptimal.<n>By combining inverse optimization geometry with conformal prediction and generative modeling, CREDO produces risk certificates that are both statistically rigorous and practically interpretable.
  arXiv  Detail & Related papers  (2025-05-19T15:24:38Z)
• Coarse Set Theory for AI Ethics and Decision-Making: A Mathematical Framework for Granular Evaluations [0.0]
  Coarse Ethics (CE) is a theoretical framework that justifies coarse-grained evaluations, such as letter grades or warning labels, as ethically appropriate under cognitive and contextual constraints.<n>This paper introduces Coarse Set Theory (CST), a novel mathematical framework that models coarse-grained decision-making using totally ordered structures and coarse partitions.<n>CST defines hierarchical relations among sets and uses information-theoretic tools, such as Kullback-Leibler Divergence, to quantify the trade-off between simplification and information loss.
  arXiv  Detail & Related papers  (2025-02-11T08:18:37Z)
• Position: AI Evaluation Should Learn from How We Test Humans [65.36614996495983]
  We argue that psychometrics, a theory originating in the 20th century for human assessment, could be a powerful solution to the challenges in today's AI evaluations.
  arXiv  Detail & Related papers  (2023-06-18T09:54:33Z)
• Robust Design and Evaluation of Predictive Algorithms under Unobserved Confounding [6.925076885159747]
  We propose a framework for robust design and evaluation of predictive algorithms.<n>We show that varying confounding assumptions substantially affects credit risk predictions and fairness evaluations across income groups.
  arXiv  Detail & Related papers  (2022-12-19T20:41:44Z)
• Reliable Off-policy Evaluation for Reinforcement Learning [53.486680020852724]
  In a sequential decision-making problem, off-policy evaluation estimates the expected cumulative reward of a target policy. We propose a novel framework that provides robust and optimistic cumulative reward estimates using one or multiple logged data.
  arXiv  Detail & Related papers  (2020-11-08T23:16:19Z)
• An Uncertainty-based Human-in-the-loop System for Industrial Tool Wear Analysis [68.8204255655161]
  We show that uncertainty measures based on Monte-Carlo dropout in the context of a human-in-the-loop system increase the system's transparency and performance. A simulation study demonstrates that the uncertainty-based human-in-the-loop system increases performance for different levels of human involvement.
  arXiv  Detail & Related papers  (2020-07-14T15:47:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.