AI Evaluation Should Measure Verification Cost, Not Correctness Alone

• URL: http://arxiv.org/abs/2608.08709v1
• Date: Sun, 09 Aug 2026 13:44:26 GMT
• Title: AI Evaluation Should Measure Verification Cost, Not Correctness Alone
• Abstract summary: We argue that current evaluation metrics overlook a critical failure mode: Verification-Cost Errors (VCEs)<n>VCEs are defined operationally, by the failure of correct identification within budget rather than by any property of the output itself.
• Score: 1.3124513975412253
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The reliability of AI generative models is typically measured by output correctness, yet in practice it depends on the effort required to verify those outputs. We argue that current evaluation metrics overlook a critical failure mode: Verification-Cost Errors (VCEs), defined as incorrect input-output pairs that a declared fraction of the verifier population fails to identify within the verification budget available in a given deployment context. Unlike standard notions of "hallucination", VCEs are defined operationally, by the failure of correct identification within budget rather than by any property of the output itself. Plausibility and authoritative presentation are hypothesised contributors to that failure, not defining conditions. To capture this asymmetry, we introduce the notion of verification cost relative to a deployment budget as an operational dimension that current evaluation does not routinely capture. The quantity is presented as a conceptual instrument rather than a finalized metric. Evidence from code generation and multi-modal document understanding shows that high benchmark accuracy can mask significant verification effort in practice. We therefore take the position that correctness alone is insufficient as a measure of reliability. AI evaluation should explicitly account for verification cost, reflecting whether errors can be detected under realistic resource constraints.

Related papers

• Prediction certification cannot replace explanation certification: a competence envelope for trustworthy AI under compound stress [0.0]
  We show that a reliable model and a compromised one can be identical under every prediction-side certificate.<n>We introduce a framework that combines prediction and explanation certification into a single deployable criterion.
  arXiv  Detail & Related papers  (2026-08-21T07:45:17Z)
• The Geometry of Flow-Matching Uncertainty: A Cost-free Uncertainty Proxy and Its Application in Flow-based VLA Failure Detection [40.15880250907137]
  Flow matching (FM) has become a popular action head paradigm for modern embodied models.<n>We show that uncertainty manifests as deviation from an ideal affine-isotropic contraction field.<n>We introduce denoising acceleration ($mathrmaccel$), a cost-free uncertainty proxy that measures the bending of the trajectory from a single forward pass.
  arXiv  Detail & Related papers  (2026-07-30T09:44:46Z)
• Bayesian control for coding agents [63.64172141184361]
  We propose a cost-sensitive sequential hypothesis testing framework for coding agents.<n>A Bayesian controller maintains a belief over correctness and decides whether to gather more evidence, refine the candidate, verify it, or stop.<n>We show that the belief state yields an interpretable correctness score that outperforms token-probability and raw tool-success baselines for uncertainty quantification.
  arXiv  Detail & Related papers  (2026-06-23T11:41:32Z)
• Conformal Reliability: A New Evaluation Metric for Conditional Generation [62.761166941396844]
  Conditional generative models have recently remarkable success in various applications.<n>We propose a novel evaluation metric called reliability based on prediction set at a pre-specified confidence level.<n>We introduce Conformal ReLiability (CReL), a framework that can (i) construct a prediction set and (ii) accurately optimize the reliability score within the constructed prediction set.
  arXiv  Detail & Related papers  (2026-05-29T03:52:44Z)
• Towards Generation-Efficient Uncertainty Estimation in Large Language Models [6.111244917760012]
  Uncertainty estimation is important for deploying LLMs in high-stakes applications such as healthcare and finance.<n>Existing methods require one or more full autoregressive generations to estimate uncertainty.<n>We investigate whether effective uncertainty estimation can be achieved with partial generation or even input-only information.
  arXiv  Detail & Related papers  (2026-05-07T11:39:13Z)
• Justified or Just Convincing? Error Verifiability as a Dimension of LLM Quality [24.614192550852277]
  No standard measure exists for whether model-generated justifications help users distinguish correct answers from incorrect ones.<n>We formalize this idea as error verifiability and propose $v_textbal$, a balanced metric that measures whether justifications enable raters to accurately assess answer correctness.<n>We introduce two methods that succeed at improving verifiability: reflect-and-rephrase (RR) for mathematical reasoning and oracle-rephrase (OR) for factual QA.
  arXiv  Detail & Related papers  (2026-04-06T04:53:59Z)
• K$α$LOS finds Consensus: A Meta-Algorithm for Evaluating Inter-Annotator Agreement in Complex Vision Tasks [4.297070083645049]
  We propose K$$LOS, a unified meta-algorithm that generalizes the "Localization First" principle to dataset quality evaluation.<n>By resolving spatial correspondence before assessing agreement, our framework transforms complex-categorical problems into nominal reliability.
  arXiv  Detail & Related papers  (2026-03-28T08:54:05Z)
• On Calibration of Large Language Models: From Response To Capability [66.59139960234326]
  Large language models (LLMs) are widely deployed as general-purpose problem solvers.<n>We introduce capability calibration, which targets the model's expected accuracy on a query.<n>Our results demonstrate that capability-calibrated confidence improves pass@$k$ prediction and inference budget allocation.
  arXiv  Detail & Related papers  (2026-02-14T01:07:45Z)
• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• Uncertainty in Language Models: Assessment through Rank-Calibration [65.10149293133846]
  Language Models (LMs) have shown promising performance in natural language generation. It is crucial to correctly quantify their uncertainty in responding to given inputs. We develop a novel and practical framework, termed $Rank$-$Calibration$, to assess uncertainty and confidence measures for LMs.
  arXiv  Detail & Related papers  (2024-04-04T02:31:05Z)
• Evaluating AI systems under uncertain ground truth: a case study in dermatology [43.8328264420381]
  We show that ignoring uncertainty leads to overly optimistic estimates of model performance.<n>In skin condition classification, we find that a large portion of the dataset exhibits significant ground truth uncertainty.
  arXiv  Detail & Related papers  (2023-07-05T10:33:45Z)
• Approaching Neural Network Uncertainty Realism [53.308409014122816]
  Quantifying or at least upper-bounding uncertainties is vital for safety-critical systems such as autonomous vehicles. We evaluate uncertainty realism -- a strict quality criterion -- with a Mahalanobis distance-based statistical test. We adopt it to the automotive domain and show that it significantly improves uncertainty realism compared to a plain encoder-decoder model.
  arXiv  Detail & Related papers  (2021-01-08T11:56:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.