From Accuracy to Readiness: Metrics and Benchmarks for Human-AI Decision-Making

• URL: http://arxiv.org/abs/2603.18895v1
• Date: Thu, 19 Mar 2026 13:35:29 GMT
• Title: From Accuracy to Readiness: Metrics and Benchmarks for Human-AI Decision-Making
• Abstract summary: Empirical evidence shows that many failures arise from miscalibrated reliance.<n>This paper proposes a measurement framework for evaluating human-AI decision-making centered on team readiness.
• Score: 0.5153774021264936
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Artificial intelligence (AI) systems are deployed as collaborators in human decision-making. Yet, evaluation practices focus primarily on model accuracy rather than whether human-AI teams are prepared to collaborate safely and effectively. Empirical evidence shows that many failures arise from miscalibrated reliance, including overuse when AI is wrong and underuse when it is helpful. This paper proposes a measurement framework for evaluating human-AI decision-making centered on team readiness. We introduce a four part taxonomy of evaluation metrics spanning outcomes, reliance behavior, safety signals, and learning over time, and connect these metrics to the Understand-Control-Improve (U-C-I) lifecycle of human-AI onboarding and collaboration. By operationalizing evaluation through interaction traces rather than model properties or self-reported trust, our framework enables deployment-relevant assessment of calibration, error recovery, and governance. We aim to support more comparable benchmarks and cumulative research on human-AI readiness, advancing safer and more accountable human-AI collaboration.

Related papers

• A Human-in-the-Loop Framework for AI-Assisted Scoring in Large-Scale Writing Assessment [0.0]
  This study presents the design and validation of an AI-assisted scoring framework for written responses in a large-scale national assessment.<n>The proposed approach focuses on short written texts of approximately 150-200 words and incorporates a human-in-the-loop strategy to preserve assessment quality while reducing manual workload.
  arXiv  Detail & Related papers  (2026-09-04T13:47:29Z)
• Robust Human-AI Complementarity under Uncertainty [16.894647212197782]
  We study how asymmetric information about the quality of information available to a human decision maker vs. an AI impacts the ability of a decision maker to extract complementary value from AI predictions.<n>We show that a key factor is the error correlation structure between human and AI predictions.<n>In particular, when the AI's prediction errors are textitnegatively correlated with those of the human, the decision maker can construct robust strategies which guarantee improvements in expected utility.
  arXiv  Detail & Related papers  (2026-07-07T17:58:06Z)
• Epistemology gives a Future to Complementarity in Human-AI Interactions [42.371764229953165]
  complementarity is the claim that a human supported by an AI system can outperform either alone in a decision-making process.<n>We argue that historical instances of complementarity function as evidence that a given human-AI interaction is a reliable process.
  arXiv  Detail & Related papers  (2026-01-14T21:04:28Z)
• How can we assess human-agent interactions? Case studies in software agent design [52.953425368394306]
  We make two major steps towards the rigorous assessment of human-agent interactions.<n>We propose PULSE, a framework for more efficient human-centric evaluation of agent designs.<n>We deploy the framework on a large-scale web platform built around the open-source software agent OpenHands.
  arXiv  Detail & Related papers  (2025-10-10T19:04:28Z)
• The AI Imperative: Scaling High-Quality Peer Review in Machine Learning [49.87236114682497]
  We argue that AI-assisted peer review must become an urgent research and infrastructure priority.<n>We propose specific roles for AI in enhancing factual verification, guiding reviewer performance, assisting authors in quality improvement, and supporting ACs in decision-making.
  arXiv  Detail & Related papers  (2025-06-09T18:37:14Z)
• When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaboration [79.69935257008467]
  We introduce Knowledge Integration and Transfer Evaluation (KITE), a conceptual and experimental framework for Human-AI knowledge transfer capabilities.<n>We conduct the first large-scale human study (N=118) explicitly designed to measure it.<n>In our two-phase setup, humans first ideate with an AI on problem-solving strategies, then independently implement solutions, isolating model explanations' influence on human understanding.
  arXiv  Detail & Related papers  (2025-06-05T20:48:16Z)
• On Benchmarking Human-Like Intelligence in Machines [77.55118048492021]
  We argue that current AI evaluation paradigms are insufficient for assessing human-like cognitive capabilities.<n>We identify a set of key shortcomings: a lack of human-validated labels, inadequate representation of human response variability and uncertainty, and reliance on simplified and ecologically-invalid tasks.
  arXiv  Detail & Related papers  (2025-02-27T20:21:36Z)
• To Err Is AI! Debugging as an Intervention to Facilitate Appropriate Reliance on AI Systems [11.690126756498223]
  Vision for optimal human-AI collaboration requires 'appropriate reliance' of humans on AI systems. In practice, the performance disparity of machine learning models on out-of-distribution data makes dataset-specific performance feedback unreliable.
  arXiv  Detail & Related papers  (2024-09-22T09:43:27Z)
• A Decision Theoretic Framework for Measuring AI Reliance [20.669176502049066]
  Humans frequently make decisions with the aid of artificially intelligent (AI) systems.<n>Researchers have identified ensuring that a human has appropriate reliance on an AI as a critical component of achieving complementary performance.<n>We propose a formal definition of reliance, based on statistical decision theory, which separates the concepts of reliance as the probability the decision-maker follows the AI's recommendation.
  arXiv  Detail & Related papers  (2024-01-27T09:13:09Z)
• Confounding-Robust Policy Improvement with Human-AI Teams [8.315707564931465]
  We propose a novel solution to address unobserved confounding in human-AI collaboration.<n>Our approach combines domain expertise with AI-driven statistical modeling to account for potentially hidden confounders.
  arXiv  Detail & Related papers  (2023-10-13T02:39:52Z)
• Position: AI Evaluation Should Learn from How We Test Humans [65.36614996495983]
  We argue that psychometrics, a theory originating in the 20th century for human assessment, could be a powerful solution to the challenges in today's AI evaluations.
  arXiv  Detail & Related papers  (2023-06-18T09:54:33Z)
• Advancing Human-AI Complementarity: The Impact of User Expertise and Algorithmic Tuning on Joint Decision Making [10.890854857970488]
  Many factors can impact success of Human-AI teams, including a user's domain expertise, mental models of an AI system, trust in recommendations, and more. Our study examined user performance in a non-trivial blood vessel labeling task where participants indicated whether a given blood vessel was flowing or stalled. Our results show that while recommendations from an AI-Assistant can aid user decision making, factors such as users' baseline performance relative to the AI and complementary tuning of AI error types significantly impact overall team performance.
  arXiv  Detail & Related papers  (2022-08-16T21:39:58Z)
• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making [53.62514158534574]
  We study whether features that reveal case-specific model information can calibrate trust and improve the joint performance of the human and AI. We show that confidence score can help calibrate people's trust in an AI model, but trust calibration alone is not sufficient to improve AI-assisted decision making.
  arXiv  Detail & Related papers  (2020-01-07T15:33:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.