Characterizing Faults in Agentic AI: A Taxonomy of Types, Symptoms, and Root Causes

• URL: http://arxiv.org/abs/2603.06847v1
• Date: Fri, 06 Mar 2026 20:12:29 GMT
• Title: Characterizing Faults in Agentic AI: A Taxonomy of Types, Symptoms, and Root Causes
• Abstract summary: Agentic AI systems combine large language model (LLM) reasoning with external invocation tool and long-horizon task execution.<n>There is limited empirical understanding of how faults originate, manifest, and propagate in real-world agentic AI systems.
• Score: 6.568786173861302
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agentic AI systems combine large language model (LLM) reasoning with external tool invocation and long-horizon task execution. Although these systems are increasingly deployed in practice, their architectural composition introduces reliability challenges that differ from those in traditional software systems and standalone LLM applications. However, there is limited empirical understanding of how faults originate, manifest, and propagate in real-world agentic AI systems. To address this gap, we conduct a large-scale empirical study of faults in agentic AI systems. We collect 13,602 issues and pull requests from 40 open-source agentic AI repositories and apply stratified sampling to select 385 faults for in-depth qualitative analysis. Using grounded theory, we derive taxonomies of fault types, observable symptoms, and root causes. We further apply Apriori-based association rule mining to identify statistically significant relationships among faults, symptoms, and root causes, revealing common fault propagation patterns. Finally, we validate the taxonomy through a developer study with 145 practitioners. Our analysis identifies 37 distinct fault types grouped into 13 higher-level fault categories, along with 13 classes of observable symptoms and 12 categories of root causes. The results show that many failures originate from mismatches between probabilistically generated artifacts and deterministic interface constraints, frequently involving dependency integration, data validation, and runtime environment handling. Association rule mining further reveals recurring propagation pathways across system components, such as token management faults leading to authentication failures and datetime handling defects causing scheduling anomalies. Practitioners rated the taxonomy as representative of real-world failures (mean = 3.97/5), and 83.8% reported that it covered faults they had encountered.

Related papers

• How Far Can Root Cause Analysis Go on Real-World Telemetry Data? [0.8594140167290097]
  We show that classical causal discovery methods and existing LLM-based multi-agent systems fail to reliably identify root causes.<n>We present a Structured Multi-Agent RCA pipeline that substantially outperforms existing LLM-based and classical baselines.
  arXiv  Detail & Related papers  (2026-07-15T07:54:46Z)
• FLARE-AI: Flaw Reporting for AI [51.64618567247884]
  Flaw reporting for deployed AI systems is fundamental to identifying system failures and improving AI safety.<n>Yet the AI reporting ecosystem is fragmented: researchers who identify flaws often do not know what to report, and groups who receive reports rarely share them with other relevant stakeholders.<n>We audit 12 reporting systems published by AI developers, cybersecurity groups, and AI flaw aggregators.<n>We introduce FLARE-AI, an open-source AI flaw reporting system designed for interoperability with existing systems.
  arXiv  Detail & Related papers  (2026-06-30T12:27:09Z)
• VerifyMAS: Hypothesis Verification for Failure Attribution in LLM Multi-Agent Systems [79.51005192758262]
  Large language model-driven multi-agent systems excel at complex tasks.<n>However, unreliable agents remain a key bottleneck to system-level reliability.<n>We propose a hypothesis verification framework for agent failure attribution.
  arXiv  Detail & Related papers  (2026-05-17T14:09:35Z)
• MAS-FIRE: Fault Injection and Reliability Evaluation for LLM-Based Multi-Agent Systems [38.44649280816596]
  We propose MAS-FIRE, a systematic framework for fault injection and reliability evaluation of Multi-Agent Systems.<n>We define a taxonomy of 15 fault types covering intra-agent cognitive errors and inter-agent coordination failures.<n>Applying MAS-FIRE to three representative MAS architectures, we uncover a rich set of fault-tolerant behaviors.
  arXiv  Detail & Related papers  (2026-02-23T13:47:43Z)
• Diagnosing Failure Root Causes in Platform-Orchestrated Agentic Systems: Dataset, Taxonomy, and Benchmark [23.342903884925576]
  This paper presents a study of root cause identification of platform-orchestrated agentic systems.<n>We construct a dataset AgentFail containing 307 failure logs from ten agentic systems, each with fine-grained annotations linking failures to their root causes.<n>We develop a taxonomy that characterizes failure root causes and analyze their distribution across different platforms and task domains.
  arXiv  Detail & Related papers  (2025-09-28T08:30:03Z)
• Why Attention Fails: A Taxonomy of Faults in Attention-Based Neural Networks [3.261740322943355]
  We present the first comprehensive empirical study of faults in attention-based neural networks (ABNNs)<n>Our work is based on a systematic analysis of 555 real-world faults collected from 96 projects across ten frameworks, including GitHub, Hugging Face, and Stack Overflow.<n>Our results show that over half of the ABNN faults arise from mechanisms unique to attention architectures.
  arXiv  Detail & Related papers  (2025-08-06T23:20:18Z)
• Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems [50.29939179830491]
  Failure attribution in LLM multi-agent systems remains underexplored and labor-intensive.<n>We develop and evaluate three automated failure attribution methods, summarizing their corresponding pros and cons.<n>The best method achieves 53.5% accuracy in identifying failure-responsible agents but only 14.2% in pinpointing failure steps.
  arXiv  Detail & Related papers  (2025-04-30T23:09:44Z)
• FaultProfIT: Hierarchical Fault Profiling of Incident Tickets in Large-scale Cloud Systems [35.310727641258715]
  We propose an automated approach, named FaultProfIT, for Fault pattern Profiling of Incident Tickets. It leverages hierarchy-guided contrastive learning to train a hierarchy-aware incident encoder and predicts fault patterns with enhanced incident representations. To date, FaultProfIT has analyzed 10,000+ incidents from 30+ cloud services, successfully revealing several fault trends that have informed system improvements.
  arXiv  Detail & Related papers  (2024-02-27T15:14:19Z)
• KGroot: Enhancing Root Cause Analysis through Knowledge Graphs and Graph Convolutional Neural Networks [14.336830860792707]
  KGroot uses event knowledge and the correlation between events to perform root cause reasoning. Experiments demonstrate KGroot can locate the root cause with accuracy of 93.5% top 3 potential causes in second-level.
  arXiv  Detail & Related papers  (2024-02-11T10:30:38Z)
• Multi-modal Causal Structure Learning and Root Cause Analysis [67.67578590390907]
  We propose Mulan, a unified multi-modal causal structure learning method for root cause localization. We leverage a log-tailored language model to facilitate log representation learning, converting log sequences into time-series data. We also introduce a novel key performance indicator-aware attention mechanism for assessing modality reliability and co-learning a final causal graph.
  arXiv  Detail & Related papers  (2024-02-04T05:50:38Z)
• Progressing from Anomaly Detection to Automated Log Labeling and Pioneering Root Cause Analysis [53.24804865821692]
  This study introduces a taxonomy for log anomalies and explores automated data labeling to mitigate labeling challenges. The study envisions a future where root cause analysis follows anomaly detection, unraveling the underlying triggers of anomalies.
  arXiv  Detail & Related papers  (2023-12-22T15:04:20Z)
• Causality-Based Multivariate Time Series Anomaly Detection [63.799474860969156]
  We formulate the anomaly detection problem from a causal perspective and view anomalies as instances that do not follow the regular causal mechanism to generate the multivariate data. We then propose a causality-based anomaly detection approach, which first learns the causal structure from data and then infers whether an instance is an anomaly relative to the local causal mechanism. We evaluate our approach with both simulated and public datasets as well as a case study on real-world AIOps applications.
  arXiv  Detail & Related papers  (2022-06-30T06:00:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.