From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents

• URL: http://arxiv.org/abs/2606.09863v1
• Date: Mon, 01 Jun 2026 00:13:23 GMT
• Title: From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents
• Abstract summary: False success is common but varies by setting: 45--48% of failures in single-control tau2-bench domains, 3% in dual-control telecom, and 75.8% among AppWorld self-assessing coding-agent trajectories with explicit status claims.<n> TF-IDF detectors achieve task-disjoint AUROC 0.83 on tau2-bench and 0.95 on AppWorld, recovering 4--8x more false successes than the best judge at the same flag rate with 3,300x lower latency.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM agents can fail silently by asserting task completion when the environment state shows otherwise. We study this failure mode, false success, across two agent benchmarks: 9,876 tau2-bench trajectories from 8 model families and 1,879 AppWorld trajectories from 4 model families with text-independent ground truth. False success is common but varies by setting: 45--48% of failures in single-control tau2-bench domains, 3% in dual-control telecom, and 75.8% among AppWorld self-assessing coding-agent trajectories with explicit status claims. LLM judges fail reliably: no configuration across 5 judges, 5 prompt strategies, and full task specifications exceeds AUROC 0.65 on tau2-bench, and the same judges reach only 0.54 AUROC on AppWorld API-call traces. Judges rely on surface completion proxies -- confident closing language in tau2-bench and coarse action-sequence volume in AppWorld -- rather than verified state changes. Lightweight TF-IDF detectors achieve task-disjoint AUROC 0.83 on tau2-bench and 0.95 on AppWorld, recovering 4--8x more false successes than the best judge at the same flag rate with 3,300x lower latency. These results suggest that production monitoring should use lightweight, domain-calibrated detectors as triage signals rather than relying on LLM judges as the primary monitor for false success.

Related papers

• From Production Traffic to Post-Training: Building a Self-Hosted LLM That Covers the Corporate Request Mix [55.08143827481755]
  We consolidate traffic from over 200 internal applications onto a single model.<n>We train a separate GRPO expert per axis and merge them via two-stage SLERP.<n>The model absorbs 50% of platform traffic, 116M requests per month, at a fraction of the serving cost.
  arXiv  Detail & Related papers  (2026-09-01T17:39:26Z)
• trajectory-judge: What Outcome-Only LLM Judges Miss on Agent Trajectories [0.0]
  Outcome-only evaluation is the production default for LLM agents.<n>We measure that blind spot where ground truth is known by construction.<n>We argue that judge evaluations must stratify recall by outcome survival.
  arXiv  Detail & Related papers  (2026-08-29T10:14:05Z)
• CURA: Certified Runtime Alarms for Computer-Use Agents [6.896160570537455]
  Self-report is the cheapest oversight channel a deployer has, and on capable computer-use agents it fails precisely where oversight matters.<n>On 361 OSWorld tasks our pipeline, a read-only feasibility gate, a planner, and an executor, reaches a mean task score of 82.9, above the 72.4 human reference.<n>64 of its 71 failures (90%) end with a success claim, acknowledging no explicit failure affordance is never used in roughly 9,100 calls.<n>Alarm-gated mid-execution oversight recovers 23 of 70 failures while spending a frontier overseer on 38, giving a deployable
  arXiv  Detail & Related papers  (2026-08-28T00:50:47Z)
• The Recall Trap: A Recall-Maximizing Retriever Configuration Reduces Issue Resolution in Fixed-Budget Code Context [45.88028371034407]
  We report a controlled case study in code repair, not a new phenomenon.<n>We toggle one flag (one-chunk-perfile deduplication) on an otherwise identical stack.<n>We map where it holds: it reverses on a lexical BM25 (3.2pp, significant cross-paradigm interaction)<n>We show that at a tight fixed budget, do not hard-deduplicate by file, and A/B packing policies against the task, not the metric flag was tuned.
  arXiv  Detail & Related papers  (2026-08-14T19:22:50Z)
• Real-Time Detection and Repair of LLM Agent Failures [0.0]
  LLM agents fail mid-episode loop, cascade tool errors, drift off goal, fabricate results, or silently absorb corrupted content.<n>We ask how much detection is achievable from observable step alone, using monitors costing microseconds per step and trained only on healthy runs.<n>System runs at 200 microseconds per step, three orders of magnitude below a judge call.
  arXiv  Detail & Related papers  (2026-08-03T16:34:46Z)
• AgentDebugX: An Open-Source Toolkit for Failure Observability, Attribution, and Recovery in LLM Agents [83.37969790806069]
  LLM agent failures are difficult to debug because the step where an error surfaces is often not the one that caused it.<n>We present Agent DebuggerX, an open-source framework that organizes debug as a closed loop of Detect, Attribute, Recover, and Rerun.<n>At its core, Deep Debugger performs multi-turn root-cause diagnosis through global trajectory understanding.
  arXiv  Detail & Related papers  (2026-07-21T06:21:13Z)
• SieveFL: Hierarchical Runtime-Aware Pruning for Scalable LLM-Based Fault Localization [0.0]
  Automated fault localization requires connecting an observed test failure to the responsible method across thousands of candidates.<n>We present SieveFL, a five-stage hierarchical framework that resolves this tension through aggressive pre-LLM filtering.
  arXiv  Detail & Related papers  (2026-05-13T13:16:41Z)
• VERDI: Single-Call Confidence Estimation for Verification-Based LLM Judges via Decomposed Inference [0.0]
  VERDI (VERification-Decomposed Inference Inference) is a method that extracts confidence from a structured judge.<n> VERDI decomposes each evaluation into sub-checks and derives three structural signals.<n>On three public benchmarks, VERDI achieves AUROC 0.72-0.91 on GPT-4.1-mini and 0.66-0.80 on GPT-5.4-mini.
  arXiv  Detail & Related papers  (2026-05-11T23:39:19Z)
• Governance Architecture for Autonomous Agent Systems: Threats, Framework, and Engineering Practice [0.0]
  We propose the Layered Governance Architecture (LGA), a four-layer framework comprising execution sandboxing (L1), intent verification (L2), zero-trust inter-agent authorization (L3), and immutable audit logging (L4)<n>To evaluate LGA, we construct a benchmark (Chinese original, English via machine translation) of 1,081 tool-call samples -- covering prompt injection, RAG poisoning, and malicious skill plugins -- and apply it to OpenClaw, a representative open-source agent framework.
  arXiv  Detail & Related papers  (2026-03-07T13:05:14Z)
• Where LLM Agents Fail and How They can Learn From Failures [62.196870049524364]
  Large Language Model (LLM) agents have shown promise in solving complex, multi-step tasks.<n>They amplify vulnerability to cascading failures, where a single root-cause error propagates through subsequent decisions.<n>Current systems lack a framework that can comprehensively understand agent error in a modular and systemic way.<n>We introduce the AgentErrorTaxonomy, a modular classification of failure modes spanning memory, reflection, planning, action, and system-level operations.
  arXiv  Detail & Related papers  (2025-09-29T18:20:27Z)
• Impact of Labeling Inaccuracy and Image Noise on Tooth Segmentation in Panoramic Radiographs using Federated, Centralized and Local Learning [46.232038247686745]
  Federated learning (FL) may mitigate privacy constraints, heterogeneous data quality, and inconsistent labeling in dental diagnostic AI.<n>We compared FL with centralized (CL) and local learning (LL) for tooth segmentation in panoramic radiographs across multiple data corruption scenarios.
  arXiv  Detail & Related papers  (2025-09-08T11:07:47Z)
• ObjexMT: Objective Extraction and Metacognitive Calibration for LLM-as-a-Judge under Multi-Turn Jailbreaks [12.396822247035578]
  We present exMT, a benchmark for objective extraction and metacognition.<n>Given a multi-turn transcript, a model must output a one-sentence base objective and a self-reported confidence.<n> Accuracy is scored by similarity to gold objectives, then thresholded once on 300 calibration items.
  arXiv  Detail & Related papers  (2025-08-23T03:32:04Z)
• Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems [50.29939179830491]
  Failure attribution in LLM multi-agent systems remains underexplored and labor-intensive.<n>We develop and evaluate three automated failure attribution methods, summarizing their corresponding pros and cons.<n>The best method achieves 53.5% accuracy in identifying failure-responsible agents but only 14.2% in pinpointing failure steps.
  arXiv  Detail & Related papers  (2025-04-30T23:09:44Z)
• Inference-Time Decontamination: Reusing Leaked Benchmarks for Large Language Model Evaluation [61.350306618479365]
  Leakage of benchmarks can prevent the accurate assessment of large language models' true performance. We propose Inference-Time Decontamination (ITD) to address this issue. ITD reduces inflated accuracy by 22.9% on GSM8K and 19.0% on MMLU.
  arXiv  Detail & Related papers  (2024-06-20T04:35:59Z)
• LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond [135.8013388183257]
  We propose a new protocol for inconsistency detection benchmark creation and implement it in a 10-domain benchmark called SummEdits. Most LLMs struggle on SummEdits, with performance close to random chance. The best-performing model, GPT-4, is still 8% below estimated human performance.
  arXiv  Detail & Related papers  (2023-05-23T21:50:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.