The Checking Problem: What must be true before AI ships in a regulated firm

• URL: http://arxiv.org/abs/2607.28666v1
• Date: Sat, 25 Jul 2026 02:35:12 GMT
• Title: The Checking Problem: What must be true before AI ships in a regulated firm
• Abstract summary: Six document-heavy of the kind performed daily in regulated financial services were run across four model families and three tool families.<n>The paper computes the review burden each configuration imposes, estimated out of sample rather than with hindsight.<n>The practical implication is that the value of an AI workflow is set less by how often it is right than by how much of it a human must still check.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Enterprise AI programmes stall at a rate that is widely quoted and poorly explained. This paper measures the mechanism. Six document-heavy workflows of the kind performed daily in regulated financial services were run across four model families and three tool configurations, three times each, producing 5,093 scored output elements across 72 configurations. Each configuration was assessed twice: against a demonstration bar, being a single correct run on a single case, and against a production bar requiring sustained accuracy, reproducibility across repeats, verifiable attribution, and a confidence signal that carries information. 57 of 72 configurations cleared the demonstration bar and 32 cleared the production bar, a survival rate of 56.1%. The paper then computes the review burden each configuration imposes, estimated out of sample rather than with hindsight. A tool that states no confidence requires review of 100% of its output, because it offers a reviewer no basis for triage. Requiring the tool to cite its sources and state a confidence reduces that to 49% while holding the residual error tolerance in 17 of 20 configurations. Adding a self-verification pass costs 2.3 times the latency of the plain configuration, reaches 44%, and is the only configuration that fails to hold the error tolerance. The practical implication is that the value of an AI workflow is set less by how often it is right than by how much of it a human must still check, and that the second property is measurable and rarely measured.

Related papers

• Fidelity Is Not Enough: Dispatch-Level Instrumentation for Agentic Datasheet Extraction [6.419341495897254]
  We log every tool call in an agentic benchmark of 25 hand-curated claims over three components, with 12 more on a fourth, 37 in all.<n>We build two instruments: a rule-based failure-attribution classifier, and a silent-failure detector whose two rules check only which tools were called, never the extracted value.
  arXiv  Detail & Related papers  (2026-08-28T15:25:12Z)
• Agent Mesh: Reliability Primitives for Non-Idempotent Agent Delegation - Identity Adequacy and Evidence Adequacy [0.0]
  We report a failure study of a production agentic software-delivery platform over 147 numbered incidents spanning 81 runs.<n>We quantify the consequences: a loop of fifty-four consecutive successful tool calls no error-rate breaker could see.<n>We derive seven reliability primitives whose enforcement unit is the delegation rather than the message.
  arXiv  Detail & Related papers  (2026-08-26T15:38:21Z)
• A Few Pages of Markdown: Committed AI Configuration and Lower Quality Cost after Coding-Agent Adoption [33.31789388282026]
  Coding agents increase development velocity but also technical debt.<n>Prior work reports only average effects across adopters, hiding wide differences between teams.<n>We introduce RAMP, a four-level cumulative maturity model grounded in version-controlled artifacts that teams commit to configure AI tools.
  arXiv  Detail & Related papers  (2026-08-26T00:00:48Z)
• Agents That Certify Their Own Exploits: Confidence-Scheduled Restricted Responses for Safe Opponent Exploitation [38.06569764716213]
  We introduce emphbudget-constrained confidence-scheduled restricted responses (CS-RNR)<n>CS-RNR is the first opponent-exploitation method whose safety guarantee is a certificate the agent computes on the strategy it actually deploys.
  arXiv  Detail & Related papers  (2026-07-30T16:57:57Z)
• Looping Is Not Reliability: State-Bound Evidence and Typed Revision Contracts for Agentic Code Repair [36.56438114281786]
  We study the gap between finding a correct patch and retaining, verifying, and submitting it.<n>We derive an evidence-bound typed loop contract and instantiate its mechanically enforceable subset in a reference implementation.
  arXiv  Detail & Related papers  (2026-07-27T16:05:23Z)
• TRACE-CTI: Auditable Post-Extraction Governance of TTP Claims with Knowledge Graphs [4.960649631468518]
  TRACE-CTI is a post-extraction claim-governance framework.<n>It aggregates run-level Predictions into configuration-level GraphAssertions.<n>It materializes setup-deduplicated corroboration as ConsensusAssertions.
  arXiv  Detail & Related papers  (2026-07-27T15:33:18Z)
• Confidence-Aware Tool Orchestration for Robust Video Understanding [30.37722671405153]
  We propose Robust-TO, an agentic video understanding framework that integrates per-frame trustworthiness into every stage of reasoning.<n>On two video reasoning benchmarks spanning eight tasks, Robust-TO achieves 56.4% average accuracy on clean inputs.<n>Under five realistic corruption types, Robust-TO maintains 54.3% average accuracy, 5.8%p above the strongest open-source baseline.
  arXiv  Detail & Related papers  (2026-06-25T11:37:51Z)
• Hierarchical Online Prompt Mutation with Dual-Loop Feedback for Guardrailed Evidence Document Generation: A Production-Evaluation Case Study [0.0]
  HOPM is a hierarchical online prompt mutation framework evaluated on a real marketplace dispute-evidence workflow.<n>The paper includes control setup, sample sizes, confidence intervals, paired tests, prompt-token categories, pseudocode, schema, rubric, guardrail taxonomy, and a constructed example.
  arXiv  Detail & Related papers  (2026-05-31T22:17:44Z)
• Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning [40.342912574072024]
  Large Language Models produce structured outputs such as travel plans or code solutions.<n>Individual reasoning steps may appear correct while the output as a whole violates budgets, fails test cases, or contradicts earlier deductions.<n>We propose a learned quality scorer with deterministic analytical constraint penalties for verifying structured LLM outputs.
  arXiv  Detail & Related papers  (2026-05-15T17:08:27Z)
• TRUST: A Framework for Decentralized AI Service v.0.1 [47.384270414446604]
  Large Reasoning Models (LRMs) and Multi-Agent Systems (MAS) in high-stakes domains demand reliable verification.<n>We introduce TRUST (Transparent, Robust, and Unified Services for Trustworthy AI), a decentralized framework with three innovations.<n>We prove a Safety-Profitability Theorem ensuring honest auditors profit while malicious actors incur losses.
  arXiv  Detail & Related papers  (2026-04-29T19:32:58Z)
• Claw-Eval: Toward Trustworthy Evaluation of Autonomous Agents [66.97968363332465]
  We introduce Claw-Eval, an end-to-end evaluation suite addressing all three gaps in agent benchmarks.<n>Claw-Eval comprises 300 human-verified tasks spanning 9 categories across three groups.<n>Every agent action is recorded through three independent evidence channels.
  arXiv  Detail & Related papers  (2026-04-07T17:43:18Z)
• Replayable Financial Agents: A Determinism-Faithfulness Assurance Harness for Tool-Using LLM Agents [0.7699235580548228]
  LLM agents struggle with regulatory audit replay: when asked to reproduce a transaction flagged decision with identical inputs, most deployments fail to return consistent results.<n>This paper introduces the DeterminismFaithfulness Assurance Harness (DFAH), a framework for measuring trajectory determinism and evidence-conditioned faithfulness in tool-using agents deployed in financial services.
  arXiv  Detail & Related papers  (2026-01-17T19:47:55Z)
• Shrinking the Generation-Verification Gap with Weak Verifiers [42.538675831498715]
  Verifiers can improve language model capabilities by scoring and ranking responses from generated candidates.<n>Weaver is a framework for designing a strong verifier by combining multiple weak, imperfect verifiers.
  arXiv  Detail & Related papers  (2025-06-22T23:38:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.