Case Studies of Causal Discovery from IT Monitoring Time Series
- URL: http://arxiv.org/abs/2307.15678v1
- Date: Fri, 28 Jul 2023 17:13:00 GMT
- Title: Case Studies of Causal Discovery from IT Monitoring Time Series
- Authors: Ali A\"it-Bachir, Charles K. Assaad, Christophe de Bignicourt, Emilie
Devijver, Simon Ferreira, Eric Gaussier, Hosein Mohanna, Lei Zan
- Abstract summary: The interest in causal discovery is growing in IT monitoring systems.
Applying causal discovery algorithms on IT monitoring data poses challenges.
This paper presents case studies on applying causal discovery algorithms to different IT monitoring datasets.
- Score: 3.2688234165632504
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Information technology (IT) systems are vital for modern businesses, handling
data storage, communication, and process automation. Monitoring these systems
is crucial for their proper functioning and efficiency, as it allows collecting
extensive observational time series data for analysis. The interest in causal
discovery is growing in IT monitoring systems as knowing causal relations
between different components of the IT system helps in reducing downtime,
enhancing system performance and identifying root causes of anomalies and
incidents. It also allows proactive prediction of future issues through
historical data analysis. Despite its potential benefits, applying causal
discovery algorithms on IT monitoring data poses challenges, due to the
complexity of the data. For instance, IT monitoring data often contains
misaligned time series, sleeping time series, timestamp errors and missing
values. This paper presents case studies on applying causal discovery
algorithms to different IT monitoring datasets, highlighting benefits and
ongoing challenges.
Related papers
- Progressing from Anomaly Detection to Automated Log Labeling and
Pioneering Root Cause Analysis [53.24804865821692]
This study introduces a taxonomy for log anomalies and explores automated data labeling to mitigate labeling challenges.
The study envisions a future where root cause analysis follows anomaly detection, unraveling the underlying triggers of anomalies.
arXiv Detail & Related papers (2023-12-22T15:04:20Z) - A Survey of Time Series Anomaly Detection Methods in the AIOps Domain [16.92261613814882]
Internet-based services have seen remarkable success, generating vast amounts of monitored key performance indicators (KPIs)
This review offers a comprehensive overview of time series anomaly detection in Artificial Intelligence for IT operations (AIOps)
It explores future directions for real-world and next-generation time-series anomaly detection based on recent advancements.
arXiv Detail & Related papers (2023-08-01T09:13:57Z) - Correlation-aware Spatial-Temporal Graph Learning for Multivariate
Time-series Anomaly Detection [67.60791405198063]
We propose a correlation-aware spatial-temporal graph learning (termed CST-GL) for time series anomaly detection.
CST-GL explicitly captures the pairwise correlations via a multivariate time series correlation learning module.
A novel anomaly scoring component is further integrated into CST-GL to estimate the degree of an anomaly in a purely unsupervised manner.
arXiv Detail & Related papers (2023-07-17T11:04:27Z) - PULL: Reactive Log Anomaly Detection Based On Iterative PU Learning [58.85063149619348]
We propose PULL, an iterative log analysis method for reactive anomaly detection based on estimated failure time windows.
Our evaluation shows that PULL consistently outperforms ten benchmark baselines across three different datasets.
arXiv Detail & Related papers (2023-01-25T16:34:43Z) - Lightweight Automated Feature Monitoring for Data Streams [1.4658400971135652]
We propose a flexible system, Feature Monitoring (FM), that detects data drifts in such data sets.
It monitors all features that are used by the system, while providing an interpretable features ranking whenever an alarm occurs.
This illustrates how FM eliminates the need to add custom signals to detect specific types of problems and that monitoring the available space of features is often enough.
arXiv Detail & Related papers (2022-07-18T14:38:11Z) - Causality-Based Multivariate Time Series Anomaly Detection [63.799474860969156]
We formulate the anomaly detection problem from a causal perspective and view anomalies as instances that do not follow the regular causal mechanism to generate the multivariate data.
We then propose a causality-based anomaly detection approach, which first learns the causal structure from data and then infers whether an instance is an anomaly relative to the local causal mechanism.
We evaluate our approach with both simulated and public datasets as well as a case study on real-world AIOps applications.
arXiv Detail & Related papers (2022-06-30T06:00:13Z) - How Can Subgroup Discovery Help AIOps? [0.0]
We study how Subgroup Discovery can help AIOps.
This project involves both data mining researchers and practitioners from Infologic, a French software editor.
arXiv Detail & Related papers (2021-09-10T14:41:02Z) - Federated Variational Learning for Anomaly Detection in Multivariate
Time Series [13.328883578980237]
We propose an unsupervised time series anomaly detection framework in a federated fashion.
We leave the training data distributed at the edge to learn a shared Variational Autoencoder (VAE) based on Convolutional Gated Recurrent Unit (ConvGRU) model.
Experiments on three real-world networked sensor datasets illustrate the advantage of our approach over other state-of-the-art models.
arXiv Detail & Related papers (2021-08-18T22:23:15Z) - Consistency of mechanistic causal discovery in continuous-time using
Neural ODEs [85.7910042199734]
We consider causal discovery in continuous-time for the study of dynamical systems.
We propose a causal discovery algorithm based on penalized Neural ODEs.
arXiv Detail & Related papers (2021-05-06T08:48:02Z) - Causal Inference for Time series Analysis: Problems, Methods and
Evaluation [11.925605453634638]
Time series data is a collection of chronological observations which is generated by several domains such as medical and financial fields.
We focus on two causal inference tasks, i.e., treatment effect estimation and causal discovery for time series data.
arXiv Detail & Related papers (2021-02-11T03:26:11Z) - Data Mining with Big Data in Intrusion Detection Systems: A Systematic
Literature Review [68.15472610671748]
Cloud computing has become a powerful and indispensable technology for complex, high performance and scalable computation.
The rapid rate and volume of data creation has begun to pose significant challenges for data management and security.
The design and deployment of intrusion detection systems (IDS) in the big data setting has, therefore, become a topic of importance.
arXiv Detail & Related papers (2020-05-23T20:57:12Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.