The AI Evaluability Gap: The Missing Layer for Managing Risk and Sustaining Value

• URL: http://arxiv.org/abs/2606.21015v1
• Date: Fri, 19 Jun 2026 00:58:01 GMT
• Title: The AI Evaluability Gap: The Missing Layer for Managing Risk and Sustaining Value
• Abstract summary: We argue that organizations lack sufficient evidence to support high-confidence governance decisions regarding either risk or value.<n>Existing governance approaches focus primarily on properties of systems, such as safety, fairness, reliability, compliance, and value.<n>We introduce Evaluability, defined as the capability of a system to generate, maintain, and renew evidence sufficient to support high-confidence governance decisions.
• Score: 1.6328866317851185
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Organizations deploying AI face two fundamental governance challenges: managing AI risk and sustaining AI value. Both depend on evidence whose sufficiency cannot be taken for granted. We call the shared underlying challenge the AI Evaluability Gap: the condition in which organizations lack sufficient evidence to support high-confidence governance decisions regarding either risk or value. We argue that this gap reflects a category error in current practice. Existing governance approaches focus primarily on properties of systems, such as safety, fairness, reliability, compliance, and value, while paying comparatively little attention to the evidentiary foundations required to justify decisions about those properties. We further argue that AI governance encompasses both operational decisions regarding whether a system may operate and investment decisions regarding whether it merits continued organizational resources. To address this problem, we introduce Evaluability, defined as the capability of a system to generate, maintain, and renew evidence sufficient to support high-confidence governance decisions over time. We formalize governance decisions as functions of calibrated confidence Conf(D|E) and identify six properties of evaluable evidence: observability, attributability, intervenability, verifiability, calibration, and temporal validity. The framework distinguishes Operational Certification, which relies primarily on structural evidence to justify deployment decisions, from Investment Certification, which relies primarily on causal evidence to justify continued resource allocation. We argue that evidence sufficiency is a missing layer of AI governance and that closing the AI Evaluability Gap is a prerequisite for both managing risk and sustaining value in AI-enabled organizations.

Related papers

• Validity, Reliability, and Transparency in Artificial Intelligence Regulation [2.588958866512272]
  Article argues that modern AI raises concerns of construct validity, confounding, representativeness, distribution shift, and fairness trade-offs.<n>We argue that inference should function as a precondition for proportionality assessment and deployment approval.
  arXiv  Detail & Related papers  (2026-08-06T09:36:15Z)
• Towards Trustworthy Embodied Intelligence: A Systems Framework and Graded Trustworthiness Levels [53.63220028820581]
  Embodied intelligence integrates learned perception and decision making with real-time computation, control, and physical interaction.<n>We define trustworthy embodied intelligence as the sustained capacity to execute specified tasks reliably.
  arXiv  Detail & Related papers  (2026-07-28T17:50:44Z)
• Closing the AI Trust Gap: The Case for Independent Certification for Trustworthy AI [0.1490650864003705]
  We show that firms that invest seriously in safety, fairness, and oversight cannot consistently prove to consumers, regulators, and shareholders that their systems go beyond the bare minimum of compliance.<n>We propose independent, outcome-oriented certification as the connective layer that can close the trust gap.
  arXiv  Detail & Related papers  (2026-07-17T14:28:36Z)
• The AI Resilience Gap: Bringing Artificial Intelligence Inside the Operational Resilience Perimeter [0.0]
  We argue that AI adoption creates a resilience obligation distinct from, and inadequately covered by, the trustworthy AI stack.<n>We propose the AI Resilience Framework: a regime-agnostic method for bringing AI dependencies inside the operational resilience perimeter.
  arXiv  Detail & Related papers  (2026-07-08T12:54:11Z)
• Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system security [57.35851886874902]
  Agentic AI systems execute complex tasks autonomously, but their multi-step trajectories introduce new failure modes that challenge trustworthiness.<n>This survey provides a focused examination of trustworthy agentic AI through two core dimensions that are critical for high-risk deployments.<n>For each dimension, we clarify key concepts, identify where risks emerge along the agent workflow, and summarize stage-targeted mitigation strategies.
  arXiv  Detail & Related papers  (2026-05-17T10:26:37Z)
• Trustworthy AI Suffers from Invariance Conflicts and Causality is The Solution [80.98492754957466]
  Core trustworthy AI objectives, such as fairness, robustness, privacy, and explainability, are hard to achieve simultaneously.<n>This paper argues that causality is necessary to understand and balance trade-offs in performance and multiple objectives of trustworthy AI.
  arXiv  Detail & Related papers  (2026-05-04T14:26:28Z)
• High-Risk AI Systems and the Problem of Identity in the European AI Act [0.3384279376065155]
  The EU Artificial Intelligence Act (AIA) establishes a lifecycle governance regime for high-risk AI systems.<n>We show how this logic is clarified by the function+ framework of artifact identity.<n>We make the synchronic case operationally via a minimal decision flow for audit and dispute contexts.
  arXiv  Detail & Related papers  (2026-04-17T20:24:29Z)
• AI Integrity: A New Paradigm for Verifiable AI Governance [3.7184769644515896]
  AI systems increasingly shape high-stakes decisions in healthcare, law, defense, and education.<n>Existing governance paradigms -- AI Ethics, AI Safety, and AI Alignment -- share a common limitation: they evaluate outcomes rather than verifying the reasoning process itself.<n>This paper introduces AI Integrity, a concept defined as a state in which the Authority Stack of an AI system is protected from corruption, contamination, manipulation, and bias.
  arXiv  Detail & Related papers  (2026-04-13T06:45:30Z)
• Making LLMs Reliable When It Matters Most: A Five-Layer Architecture for High-Stakes Decisions [51.56484100374058]
  Current large language models (LLMs) excel in verifiable domains where outputs can be checked before action but prove less reliable for high-stakes strategic decisions with uncertain outcomes.<n>This gap, driven by mutually cognitive biases in both humans and artificial intelligence (AI) systems, threatens the defensibility of valuations and sustainability of investments in the sector.<n>This report describes a framework emerging from systematic qualitative assessment across 7 frontier-grade LLMs and 3 market-facing venture vignettes under time pressure.
  arXiv  Detail & Related papers  (2025-11-10T22:24:21Z)
• Benchmarking is Broken -- Don't Let AI be its Own Judge [22.93026946593552]
  We argue that the current laissez-faire approach to evaluating AI is unsustainable.<n>We introduce PeerBench, a community-governed, proctored evaluation blueprint.<n>Our goal is to pave the way for evaluations that can restore integrity and deliver genuinely trustworthy measures of AI progress.
  arXiv  Detail & Related papers  (2025-10-08T21:41:37Z)
• Never Compromise to Vulnerabilities: A Comprehensive Survey on AI Governance [211.5823259429128]
  We propose a comprehensive framework integrating technical and societal dimensions, structured around three interconnected pillars: Intrinsic Security, Derivative Security, and Social Ethics.<n>We identify three core challenges: (1) the generalization gap, where defenses fail against evolving threats; (2) inadequate evaluation protocols that overlook real-world risks; and (3) fragmented regulations leading to inconsistent oversight.<n>Our framework offers actionable guidance for researchers, engineers, and policymakers to develop AI systems that are not only robust and secure but also ethically aligned and publicly trustworthy.
  arXiv  Detail & Related papers  (2025-08-12T09:42:56Z)
• Beyond Explainability: The Case for AI Validation [0.0]
  We argue for a shift toward validation as a central regulatory pillar.<n> Validation, ensuring the reliability, consistency, and robustness of AI outputs, offers a more practical, scalable, and risk-sensitive alternative to explainability.<n>We propose a forward-looking policy framework centered on pre- and post-deployment validation, third-party auditing, harmonized standards, and liability incentives.
  arXiv  Detail & Related papers  (2025-05-27T06:42:41Z)
• Trustworthy AI [75.99046162669997]
  Brittleness to minor adversarial changes in the input data, ability to explain the decisions, address the bias in their training data, are some of the most prominent limitations. We propose the tutorial on Trustworthy AI to address six critical issues in enhancing user and public trust in AI systems.
  arXiv  Detail & Related papers  (2020-11-02T20:04:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.