StepFinder: A Temporal Semantic Framework for Failure Attribution in Multi-Agent Systems

• URL: http://arxiv.org/abs/2606.03467v1
• Date: Tue, 02 Jun 2026 10:45:49 GMT
• Title: StepFinder: A Temporal Semantic Framework for Failure Attribution in Multi-Agent Systems
• Abstract summary: Failure attribution is a task that aims to automatically identify the root cause step responsible for a failure.<n>Existing failure attribution methods mainly rely on LLMs to reason over original execution trajectories.<n>We propose StepFinder, a lightweight failure attribution framework.
• Score: 11.500948775496218
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM-based multi-agent systems exhibit remarkable collaborative capabilities in complex multi-step tasks. However, these systems are highly sensitive to single-step execution errors that can propagate through agent interactions and lead to cascading failures. To understand the causes of failure and improve system reliability, failure attribution has been introduced as a task that aims to automatically identify the root cause step responsible for a failure. Existing failure attribution methods mainly rely on LLMs to reason over original execution trajectories, which not only incur high inference costs and latency, but also suffer from interference caused by redundant and noisy execution logs, causing LLMs to struggle in accurately identifying the true root cause step. To address this, we propose StepFinder, a lightweight failure attribution framework. We use LLMs solely during the feature construction phase to encode execution logs into temporal semantic sequences. Subsequently, a parameter-efficient combination of temporal modeling and attention modules is applied to capture the sequential evolution and cross-step dependencies of the trajectories. Finally, the step-level error score is refined through multi-scale differences and position bias, enabling precise root cause identification. Experimental results on the Who&When benchmark demonstrate that StepFinder outperforms LLM-based methods in step-level failure attribution while achieving substantially higher inference efficiency, reducing inference time by 79% compared with the fastest LLM-based method, with no text generation overhead. Our code is available at https://github.com/taiyu-zhu/StepFinder.

Related papers

• DCFA: Dual-view Causal-inspired Attribution for Failure Reasoning in LLM-based Multi-agent Systems [61.11435469308521]
  We propose DCFA, a training-free framework for failure attribution in large language model systems.<n> DCFA integrates a global module that constructs structured causal-inspired dependency graphs from system traces.<n>Experiments show that DCFA improves step-level accuracy by up to 8.27% over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-09-04T05:29:31Z)
• Detect Before You Attribute: Cascade Failure Attribution for Multi-Agent Systems [27.57915091923977]
  DUOTRACE is a plug-and-play detection filter for LLM-based failure attribution.<n>It first detects anomalous executions and then supplies focused trajectory evidence to downstream attribution methods.<n>Experiments show that DUOTRACE improves agent-level and step-level attribution accuracy by 8.7% and 7.0%, respectively.
  arXiv  Detail & Related papers  (2026-08-30T08:17:42Z)
• Beyond LLM-Based Reasoning: Lightweight GNNs for Agent Failure Attribution [47.70578021328887]
  Large language model (LLM)-based multi-agent systems often exhibit complex failure modes.<n>We introduce AFANet, a lightweight graph-based framework that models interaction trajectories through step-level semantic signals and agent-level relationships.<n>Our results suggest that effective agent failure attribution does not require heavy reasoning and a lightweight, structured approach can achieve strong performance.
  arXiv  Detail & Related papers  (2026-08-19T06:16:09Z)
• HPFA: Hypergraph-Based Paired Failure Attribution for LLM Reasoning [15.068861766259412]
  We propose a hypergraph-based paired failure attribution framework that attributes the failure root cause by comparing the hyperedges of the targeted failure reasoning path against a reference successful path.<n>By reducing the search space, our method efficiently localizes root causes and enables scalable synthesis of attribution data.<n>Experiments on mathematical reasoning and agentic coding tasks demonstrate that HPFA can dramatically increase attribution accuracy and efficiency.
  arXiv  Detail & Related papers  (2026-08-03T10:23:48Z)
• FALAT: Tracing Failures in LLM Agent Trajectories via Dependency-Guided Search [11.638321375070047]
  We propose FALAT, a diagnostic framework for failure attribution in LLM agent trajectories.<n>We evaluate FALAT on the Who&When benchmark, which includes both algorithm-generated and hand-crafted multi-agent failure trajectories.
  arXiv  Detail & Related papers  (2026-05-30T15:11:35Z)
• From Flat Logs to Causal Graphs: Hierarchical Failure Attribution for LLM-based Multi-Agent Systems [13.588125015284257]
  We propose CHIEF, a novel framework that transforms chaotic trajectories into a structured hierarchical causal graph.<n> Experiments on Who&When benchmark show that CHIEF outperforms eight strong and state-of-the-art baselines on both agent- and step-level accuracy.
  arXiv  Detail & Related papers  (2026-02-27T06:08:42Z)
• AgentRx: Diagnosing AI Agent Failures from Execution Trajectories [9.61742219198197]
  We release a benchmark of 115 failed trajectories spanning structured API, incident management, and open-ended web/file tasks.<n>Each trajectory is annotated with a critical failure step and a category from a grounded-theory derived, cross-domain failure taxonomy.<n>We present AGENTRX, an automated domain-agnostic diagnostic framework that pinpoints the critical failure step in a failed agent trajectory.
  arXiv  Detail & Related papers  (2026-02-02T18:54:07Z)
• DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems [48.971606069204825]
  DoVer is an intervention-driven debug framework for large language model (LLM)-based multi-agent systems.<n>It augments hypothesis generation with active verification through targeted interventions.<n>DoVer flips 18-28% of failed trials into successes, achieves up to 16% milestone progress, and validates or refutes 30-60% of failure hypotheses.
  arXiv  Detail & Related papers  (2025-12-07T09:23:48Z)
• Solving a Million-Step LLM Task with Zero Errors [13.911986576836568]
  This paper describes MAKER, the first system that successfully solves a task with over one million LLM steps with zero errors.<n>The results suggest that instead of relying on continual improvement of current LLMs, massively decomposed agentic processes (MDAPs) may provide a way to efficiently solve problems at the level of organizations and societies.
  arXiv  Detail & Related papers  (2025-11-12T06:27:55Z)
• Reasoning with Confidence: Efficient Verification of LLM Reasoning Steps via Uncertainty Heads [104.9566359759396]
  We propose a lightweight alternative for step-level reasoning verification based on data-driven uncertainty scores.<n>Our findings suggest that the internal states of LLMs encode their uncertainty and can serve as reliable signals for reasoning verification.
  arXiv  Detail & Related papers  (2025-11-09T03:38:29Z)
• Automatic Failure Attribution and Critical Step Prediction Method for Multi-Agent Systems Based on Causal Inference [8.823529310904162]
  Multi-agent systems (MAS) are critical for automating complex tasks, yet their practical deployment is hampered by the challenge of failure attribution.<n>We introduce the first failure attribution framework for MAS grounded in multi-granularity causal inference.
  arXiv  Detail & Related papers  (2025-09-10T15:22:00Z)
• Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems [50.29939179830491]
  Failure attribution in LLM multi-agent systems remains underexplored and labor-intensive.<n>We develop and evaluate three automated failure attribution methods, summarizing their corresponding pros and cons.<n>The best method achieves 53.5% accuracy in identifying failure-responsible agents but only 14.2% in pinpointing failure steps.
  arXiv  Detail & Related papers  (2025-04-30T23:09:44Z)
• ToolScan: A Benchmark for Characterizing Errors in Tool-Use LLMs [77.79172008184415]
  TOOLSCAN is a new benchmark to identify error patterns in LLM output on tool-use tasks.<n>We show that even the most prominent LLMs exhibit these error patterns in their outputs.<n>Researchers can use these insights from TOOLSCAN to guide their error mitigation strategies.
  arXiv  Detail & Related papers  (2024-11-20T18:56:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.