Tracing Agentic Failure from the Flow of Success

• URL: http://arxiv.org/abs/2607.12747v1
• Date: Tue, 14 Jul 2026 13:16:14 GMT
• Title: Tracing Agentic Failure from the Flow of Success
• Abstract summary: We argue that a practical failure attribution model should be lightweight and trainable without step-level supervision on failure data.<n>We propose OAT, which casts this problem as one-class learning with neural controlled differential equations.<n>With training on only 100 successful trajectories, OAT is 200--5000 $times$ faster than prompting-based baselines.
• Score: 7.071269758592172
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Failure attribution for LLM-based agentic systems, i.e., identifying which steps in a failure trajectory caused the task to fail, is critical for debugging and improving these systems. Existing approaches either rely on prompting-based pipelines, which are computationally expensive, or require post-training on failure trajectories with step-level error annotations, which are costly to collect and difficult to scale. We argue that a practical failure attribution model should be lightweight and trainable without step-level supervision on failure data. To this end, we address unsupervised failure attribution, i.e., training exclusively on successful trajectories and identifying error steps at inference time given a failure trajectory. We propose OAT, which casts this problem as one-class learning with neural controlled differential equations, modeling the dynamical pattern of successful trajectories in latent space. At inference time, each step in a failure trajectory is assigned an anomaly score based on its deviation from the dynamics learned on successful trajectories, which is then used to form a set of error steps. With training on only 100 successful trajectories, experiments show that OAT is 200--5000 $\times$ faster than prompting-based baselines, and, at the same time, consistently outperforms them in both in-domain and out-of-distribution datasets with +20% and +7% F1 scores, respectively, demonstrating that OAT is a promising and efficient direction for diagnosing agentic system failures.

Related papers

• TrapVLA: Trapping Vision-Language-Action Models in Configured Failure Modes [65.10128145059127]
  This work introduces Configured Failure Trapping, a novel backdoor attack task against Vision-Language-Action (VLA) models.<n>We propose an effective data engine for synthesizing high-quality target trajectories and an automated suite for measuring configured-failure fidelity.<n>Experiments show that TrapVLA effectively injects configured failure modes into VLA models while largely preserving performance on clean data.
  arXiv  Detail & Related papers  (2026-08-27T03:44:49Z)
• FailForge: Distilling Procedural Competence from Persistent Failures into Code Agents [49.519184792774304]
  FailForge is an agentic framework that converts failed rollouts into training signal.<n>FailForge recovers over 26% of previously failed instances at marginal additional cost.
  arXiv  Detail & Related papers  (2026-08-09T08:22:57Z)
• HPFA: Hypergraph-Based Paired Failure Attribution for LLM Reasoning [15.068861766259412]
  We propose a hypergraph-based paired failure attribution framework that attributes the failure root cause by comparing the hyperedges of the targeted failure reasoning path against a reference successful path.<n>By reducing the search space, our method efficiently localizes root causes and enables scalable synthesis of attribution data.<n>Experiments on mathematical reasoning and agentic coding tasks demonstrate that HPFA can dramatically increase attribution accuracy and efficiency.
  arXiv  Detail & Related papers  (2026-08-03T10:23:48Z)
• Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agents [80.03824805228719]
  A major challenge in developing computer-use agents is collecting large-scale, high-quality trajectories.<n>We propose a data-centric paradigm that turns failed trajectories into agent improvements.<n>We validate this approach with the state-of-the-art OpenCUA-72B model on the OSWorld benchmark.
  arXiv  Detail & Related papers  (2026-06-30T07:44:37Z)
• FALAT: Tracing Failures in LLM Agent Trajectories via Dependency-Guided Search [11.638321375070047]
  We propose FALAT, a diagnostic framework for failure attribution in LLM agent trajectories.<n>We evaluate FALAT on the Who&When benchmark, which includes both algorithm-generated and hand-crafted multi-agent failure trajectories.
  arXiv  Detail & Related papers  (2026-05-30T15:11:35Z)
• VerifyMAS: Hypothesis Verification for Failure Attribution in LLM Multi-Agent Systems [79.51005192758262]
  Large language model-driven multi-agent systems excel at complex tasks.<n>However, unreliable agents remain a key bottleneck to system-level reliability.<n>We propose a hypothesis verification framework for agent failure attribution.
  arXiv  Detail & Related papers  (2026-05-17T14:09:35Z)
• Verified Critical Step Optimization for LLM Agents [67.05296684575445]
  Critical Step Optimization focuses preference learning on verified critical steps.<n>Method starts from failed policy trajectories rather than expert demonstrations.<n>Experiments on GAIA-Text-103 and XBench-DeepSearch show that CSO achieves 37% and 26% relative improvement over the SFT baseline.
  arXiv  Detail & Related papers  (2026-02-03T11:41:02Z)
• InT: Self-Proposed Interventions Enable Credit Assignment in LLM Reasoning [32.274434679047395]
  Outcome-reward reinforcement learning (RL) has proven effective at improving the reasoning capabilities of large language models (LLMs)<n>Standard RL assigns credit only at the level of the final answer, penalizing entire reasoning traces when the outcome is incorrect.<n>We introduce Intervention Training (InT), a training paradigm in which the model performs fine-grained credit assignment on its own reasoning traces.
  arXiv  Detail & Related papers  (2026-01-20T18:15:38Z)
• Hierarchical Vision Language Action Model Using Success and Failure Demonstrations [60.82332413442677]
  We introduce VINE, a hierarchical vision-language-action model that separates high-level reasoning from low-level control.<n>System 2 performs feasibility-guided tree search over a 2D scene-graph abstraction.<n>System 1 executes low-level actions without modifying the agent's core skills.
  arXiv  Detail & Related papers  (2025-12-03T15:58:38Z)
• Guardian: Detecting Robotic Planning and Execution Errors with Vision-Language Models [53.20969621498248]
  We propose an automatic robot failure synthesis approach that procedurally perturbs successful trajectories to generate diverse planning and execution failures.<n>We construct three new failure detection benchmarks: RLBench-Fail, BridgeDataV2-Fail, and UR5-Fail.<n>We then train Guardian, a VLM with multi-view images for detailed failure reasoning and detection.
  arXiv  Detail & Related papers  (2025-12-01T17:57:27Z)
• Automatic Failure Attribution and Critical Step Prediction Method for Multi-Agent Systems Based on Causal Inference [8.823529310904162]
  Multi-agent systems (MAS) are critical for automating complex tasks, yet their practical deployment is hampered by the challenge of failure attribution.<n>We introduce the first failure attribution framework for MAS grounded in multi-granularity causal inference.
  arXiv  Detail & Related papers  (2025-09-10T15:22:00Z)
• Validity Learning on Failures: Mitigating the Distribution Shift in Autonomous Vehicle Planning [2.3558144417896583]
  The planning problem constitutes a fundamental aspect of the autonomous driving framework. We propose Validity Learning on Failures, VL(on failure) as a remedy to address this issue. We show that VL(on failure) outperforms the state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2024-06-03T17:25:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.