Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent Failures

• URL: http://arxiv.org/abs/2609.13463v1
• Date: Fri, 11 Sep 2026 19:29:21 GMT
• Title: Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent Failures
• Abstract summary: Existing root-cause attribution methods suffer from low diagnostic accuracy as execution traces grow larger.<n>Continual Search nudges the judge, over successive turns, to keep searching for unresolved diagnostic evidence.<n>MegaRCA-Mix provides a challenging testbed of 50 human-annotated failure trials spanning long-horizon, execution-heavy tasks.
• Score: 11.365690419023425
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The increasing deployment of AI agents in long-horizon tasks yields massive execution logs. Diagnosing failures within these records is crucial for reliability, as it transforms outcome-level signals into actionable interventions. The sheer scale of the data renders human review impractical, driving the need for automated root-cause attribution (RCA). However, automated RCA methods using LLMs suffer from low diagnostic accuracy, especially as execution traces grow larger. They struggle because relevant information is often sparse, distributed across distant actions, and disconnected from the visible failure, reducing root-cause attribution to a massive search problem. Existing RCA methods typically rely on one-shot LLM judgments to diagnose failures from execution traces. While effective for shorter trajectories, these judges tend to settle on a plausible diagnosis early, leaving critical evidence in longer traces unexamined. We introduce Continual Search, an iterative framework that nudges the judge, over successive turns, to keep searching for unresolved diagnostic evidence. We evaluate Continual Search across four existing RCA benchmarks. Recognizing the lack of massive execution traces in current benchmarks, we introduce MegaRCA-Mix to evaluate RCA at scale. MegaRCA-Mix provides a challenging testbed of 50 human-annotated failure trials spanning long-horizon, execution-heavy tasks. Across multiple benchmark suites and model families, Continual Search consistently improves attribution performance. On MegaRCA-Mix, for example, it improves GPT-5.5's F1 score by more than 40\%, from $0.349$ to $0.498$. More interestingly, within the same model family, lower-tier models can even surpass their higher-tier counterparts, demonstrating that effective search supersedes raw model scale.

Related papers

• RAP: Research Attention Prediction Reveals Target-Conditioned Evidence Acquisition Biases [58.92094935632493]
  Large language models (LLMs) increasingly act as research agents, yet their ability to track shifts in research attention is difficult to evaluate.<n>We introduce Research Attention Prediction (RAP), a rolling benchmark covering 278 AI/ML fields and 1,390 episodes.<n>At each cut-off, an LLM agent searches a temporally restricted arXiv corpus and predicts the next six months' paper shares across eight frozen research directions.
  arXiv  Detail & Related papers  (2026-09-09T12:15:14Z)
• Beyond Fault Localization: A Trajectory-Level Study of LLM Agents for Microservice Root Cause Analysis [13.865176720288938]
  We use a trajectory-level framework to evaluate root cause analysis (RCA) performance.<n>We find a disconnect between answer correctness and diagnostic quality.<n>We operationalize this taxonomy as DiagGuard, a two-stage defense-in-depth architecture.
  arXiv  Detail & Related papers  (2026-08-21T17:13:45Z)
• ThinkRetrieve: Retrieval-Augmented Reasoning Traces for Test-Time Scaling [60.48279351544583]
  ThinkRetrieve is a test-time scaling framework for large reasoning models.<n>It retrieves relevant exemplars at each intermediate step and injects them directly into the thinking trace.<n>It consistently improves accuracy over standard test-time scaling.
  arXiv  Detail & Related papers  (2026-08-11T13:58:07Z)
• SearchAuditor: Auditing and Attributing Failures in Long-Horizon Search Agents [56.400332595452966]
  We introduce SearchAuditBench, a benchmark that evaluates whether auditors can localize, attribute, and repair search-agent failures.<n>SearchAuditBench comprises 1,243 failed trajectories, averaging 73.1 messages and 65.1K tokens, collected from eight open-weight models on five deep-search benchmarks.<n>We propose SearchAuditor, a multi-perspective auditing framework that effectively localizes, attributes, and repairs search-agent failures.
  arXiv  Detail & Related papers  (2026-08-05T09:09:00Z)
• AREX: Towards a Recursively Self-Improving Agent for Deep Research [75.96468303743958]
  We introduce AREX, a family of Recursively Self-Improving deep research agents.<n>AREX alternates between an inner research loop that gathers evidence and constructs a provisional answer.<n>We train AREX on verified synthetic tasks and high-quality trajectories through agentic mid-training and long-horizon reinforcement learning.
  arXiv  Detail & Related papers  (2026-07-23T16:05:46Z)
• Silent Failures in Multimodal Agentic Search:A Diagnostic Taxonomy and Cross-Judge Evaluation [7.095219140870008]
  Existing evaluations mainly focus on final-answer accuracy and may miss failures in the search trajectory.<n>We introduce a six-category taxonomy covering shortcuts, phantom grounding, wrong-evidence-right-answer cases, over-retrieval laundering, cross-modal contradiction, and provenance.<n>We build a trajectory-level diagnostic pipeline that evaluates both answer correctness and evidence-grounding quality under a unified ReAct-style scaffold.
  arXiv  Detail & Related papers  (2026-07-22T06:19:06Z)
• Anomaly Detection and Root Cause Analysis for Microservice Systems [0.0]
  Microservice systems are widely used to build cloud applications, yet their complexity makes failures inevitable.<n>This thesis advances automated anomaly detection and root cause analysis for microservice failures.
  arXiv  Detail & Related papers  (2026-06-08T00:50:43Z)
• Stalled, Biased, and Confused: Uncovering Reasoning Failures in LLMs for Cloud-Based Root Cause Analysis [5.532586951580959]
  We present a focused empirical evaluation that isolates an LLM's reasoning behavior.<n>We produce a labeled taxonomy of 16 common RCA reasoning failures and use an LLM-as-a-Judge for annotation.
  arXiv  Detail & Related papers  (2026-01-29T18:23:26Z)
• Demystifying deep search: a holistic evaluation with hint-free multi-hop questions and factorised metrics [89.1999907891494]
  We present WebDetective, a benchmark of hint-free multi-hop questions paired with a controlled Wikipedia sandbox.<n>Our evaluation of 25 state-of-the-art models reveals systematic weaknesses across all architectures.<n>We develop an agentic workflow, EvidenceLoop, that explicitly targets the challenges our benchmark identifies.
  arXiv  Detail & Related papers  (2025-10-01T07:59:03Z)
• GALA: Can Graph-Augmented Large Language Model Agentic Workflows Elevate Root Cause Analysis? [9.394057684388027]
  This paper introduces GALA, a novel framework for root cause analysis in microservice systems.<n> evaluated on an open-source benchmark, GALA achieves substantial improvements over state-of-the-art methods.<n>We show that GALA bridges the gap between automated failure diagnosis and practical incident resolution.
  arXiv  Detail & Related papers  (2025-08-17T19:12:05Z)
• Disentangled Causal Graph Learning for Online Unsupervised Root Cause Analysis [49.910053255238566]
  Root cause analysis (RCA) can identify the root causes of system faults/failures by analyzing system monitoring data. Previous research has mostly focused on developing offline RCA algorithms, which often require manually initiating the RCA process. We propose CORAL, a novel online RCA framework that can automatically trigger the RCA process and incrementally update the RCA model.
  arXiv  Detail & Related papers  (2023-05-18T01:27:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.