Causal AI-based Root Cause Identification: Research to Practice at Scale

• URL: http://arxiv.org/abs/2502.18240v1
• Date: Tue, 25 Feb 2025 14:20:33 GMT
• Title: Causal AI-based Root Cause Identification: Research to Practice at Scale
• Authors: Saurabh Jha, Ameet Rahane, Laura Shwartz, Marc Palaci-Olgun, Frank Bagehorn, Jesus Rios, Dan Stingaciu, Ragu Kattinakere, Debasish Banerjee,
• Abstract summary: We have developed a novel causality-based Root Cause Identification (RCI) algorithm that emphasizes causation over correlation.<n>This paper highlights Instana's advanced failure diagnosis capabilities, discussing both the theoretical underpinnings and practical implementations of the RCI algorithm.
• Score: 2.455633941531165
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Modern applications are built as large, distributed systems spanning numerous modules, teams, and data centers. Despite robust engineering and recovery strategies, failures and performance issues remain inevitable, risking significant disruptions and affecting end users. Rapid and accurate root cause identification is therefore vital to ensure system reliability and maintain key service metrics. We have developed a novel causality-based Root Cause Identification (RCI) algorithm that emphasizes causation over correlation. This algorithm has been integrated into IBM Instana-bridging research to practice at scale-and is now in production use by enterprise customers. By leveraging "causal AI," Instana stands apart from typical Application Performance Management (APM) tools, pinpointing issues in near real-time. This paper highlights Instana's advanced failure diagnosis capabilities, discussing both the theoretical underpinnings and practical implementations of the RCI algorithm. Real-world examples illustrate how our causality-based approach enhances reliability and performance in today's complex system landscapes.

Related papers

• Thinking Longer, Not Larger: Enhancing Software Engineering Agents via Scaling Test-Time Compute [61.00662702026523]
  We propose a unified Test-Time Compute scaling framework that leverages increased inference-time instead of larger models. Our framework incorporates two complementary strategies: internal TTC and external TTC. We demonstrate our textbf32B model achieves a 46% issue resolution rate, surpassing significantly larger models such as DeepSeek R1 671B and OpenAI o1.
  arXiv  Detail & Related papers  (2025-03-31T07:31:32Z)
• STAR: A Foundation Model-driven Framework for Robust Task Planning and Failure Recovery in Robotic Systems [5.426894918217948]
  STAR (Smart Task Adaptation and Recovery) is a novel framework that synergizes Foundation Models (FMs) with dynamically expanding Knowledge Graphs (KGs) FMs offer remarkable generalization and contextual reasoning, but their limitations hinder reliable deployment. We show that STAR demonstrated an 86% task planning accuracy and 78% recovery success rate, showing significant improvements over baseline methods.
  arXiv  Detail & Related papers  (2025-03-08T05:05:21Z)
• LLM Assisted Anomaly Detection Service for Site Reliability Engineers: Enhancing Cloud Infrastructure Resilience [5.644170923282226]
  This paper introduces a scalable Anomaly Detection Service with a generalizable API tailored for industrial time-series data.<n>We provide insights into the usage patterns of the service, with over 500 users and 200,000 API calls in a year.<n>We plan to extend the system to include time series foundation models, enabling zero-shot anomaly detection capabilities.
  arXiv  Detail & Related papers  (2025-01-28T06:41:37Z)
• Online Multi-modal Root Cause Analysis [61.94987309148539]
  Root Cause Analysis (RCA) is essential for pinpointing the root causes of failures in microservice systems. Existing online RCA methods handle only single-modal data overlooking, complex interactions in multi-modal systems. We introduce OCEAN, a novel online multi-modal causal structure learning method for root cause localization.
  arXiv  Detail & Related papers  (2024-10-13T21:47:36Z)
• Exploring LLM-based Agents for Root Cause Analysis [17.053079105858497]
  Root cause analysis (RCA) is a critical part of the incident management process. Large Language Models (LLMs) have been used to perform RCA, but are not able to collect additional diagnostic information. We present an evaluation of a ReAct agent equipped with retrieval tools, on an out-of-distribution dataset of production incidents collected at Microsoft.
  arXiv  Detail & Related papers  (2024-03-07T00:44:01Z)
• Analyzing Adversarial Inputs in Deep Reinforcement Learning [53.3760591018817]
  We present a comprehensive analysis of the characterization of adversarial inputs, through the lens of formal verification. We introduce a novel metric, the Adversarial Rate, to classify models based on their susceptibility to such perturbations. Our analysis empirically demonstrates how adversarial inputs can affect the safety of a given DRL system with respect to such perturbations.
  arXiv  Detail & Related papers  (2024-02-07T21:58:40Z)
• Multi-modal Causal Structure Learning and Root Cause Analysis [67.67578590390907]
  We propose Mulan, a unified multi-modal causal structure learning method for root cause localization. We leverage a log-tailored language model to facilitate log representation learning, converting log sequences into time-series data. We also introduce a novel key performance indicator-aware attention mechanism for assessing modality reliability and co-learning a final causal graph.
  arXiv  Detail & Related papers  (2024-02-04T05:50:38Z)
• Root Cause Analysis In Microservice Using Neural Granger Causal Discovery [12.35924469567586]
  We propose RUN, a novel approach for root cause analysis using neural Granger causal discovery with contrastive learning. RUN enhances the backbone encoder by integrating contextual information from time series, and leverages a time series forecasting model to conduct neural Granger causal discovery. In addition, RUN incorporates Pagerank with a vector to efficiently recommend the top-k root causes.
  arXiv  Detail & Related papers  (2024-02-02T04:43:06Z)
• AttNS: Attention-Inspired Numerical Solving For Limited Data Scenarios [51.94807626839365]
  We propose the attention-inspired numerical solver (AttNS) to solve differential equations due to limited data.<n>AttNS is inspired by the effectiveness of attention modules in Residual Neural Networks (ResNet) in enhancing model generalization and robustness.
  arXiv  Detail & Related papers  (2023-02-05T01:39:21Z)
• On a Uniform Causality Model for Industrial Automation [61.303828551910634]
  A Uniform Causality Model for various application areas of industrial automation is proposed. The resulting model describes the behavior of Cyber-Physical Systems mathematically. It is shown that the model can work as a basis for the application of new approaches in industrial automation that focus on machine learning.
  arXiv  Detail & Related papers  (2022-09-20T11:23:51Z)
• Accelerating Recursive Partition-Based Causal Structure Learning [4.357523892518871]
  Recursive causal discovery algorithms provide good results by using Conditional Independent (CI) tests in smaller sub-problems. This paper proposes a generic causal structure refinement strategy that can locate the undesired relations with a small number of CI-tests. We then empirically evaluate its performance against the state-of-the-art algorithms in terms of solution quality and completion time in synthetic and real datasets.
  arXiv  Detail & Related papers  (2021-02-23T08:28:55Z)
• Towards AIOps in Edge Computing Environments [60.27785717687999]
  This paper describes the system design of an AIOps platform which is applicable in heterogeneous, distributed environments. It is feasible to collect metrics with a high frequency and simultaneously run specific anomaly detection algorithms directly on edge devices.
  arXiv  Detail & Related papers  (2021-02-12T09:33:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.