Synthetic Time Series for Anomaly Detection in Cloud Microservices

• URL: http://arxiv.org/abs/2408.00006v1
• Date: Sun, 21 Jul 2024 11:23:54 GMT
• Title: Synthetic Time Series for Anomaly Detection in Cloud Microservices
• Authors: Mohamed Allam, Noureddine Boujnah, Noel E. O'Connor, Mingming Liu,
• Abstract summary: This paper proposes a framework for time series generation built to investigate anomaly detection in cloud computing. We detail the pipeline implementation that allows deployment and management of as well as the theoretical approach required to generate anomalies. Two datasets generated using the proposed framework have been made publicly available through GitHub.
• Score: 9.44541023672687
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper proposes a framework for time series generation built to investigate anomaly detection in cloud microservices. In the field of cloud computing, ensuring the reliability of microservices is of paramount concern and yet a remarkably challenging task. Despite the large amount of research in this area, validation of anomaly detection algorithms in realistic environments is difficult to achieve. To address this challenge, we propose a framework to mimic the complex time series patterns representative of both normal and anomalous cloud microservices behaviors. We detail the pipeline implementation that allows deployment and management of microservices as well as the theoretical approach required to generate anomalies. Two datasets generated using the proposed framework have been made publicly available through GitHub.

Related papers

• MicroRacer: Detecting Concurrency Bugs for Cloud Service Systems [2.1647389701624165]
  Cloud service systems are vulnerable to bugs due to their microservice architecture.<n>Existing methods for bug detection often fall short to their intrusive nature and inability to handle the architectural complexities.<n>We propose MicroRacer, a non-intrusive and automated framework for detecting bugs in such environments.
  arXiv  Detail & Related papers  (2025-12-05T13:43:31Z)
• Graph Neural AI with Temporal Dynamics for Comprehensive Anomaly Detection in Microservices [7.957284443727372]
  This study addresses the problem of anomaly detection and root cause tracing in microservice architectures.<n>It proposes a unified framework that combines graph neural networks with temporal modeling.
  arXiv  Detail & Related papers  (2025-11-05T08:28:41Z)
• Isolation-based Spherical Ensemble Representations for Anomaly Detection [60.989157958972356]
  Anomaly detection is a critical task in data mining and management with applications spanning fraud detection, network security, and log monitoring.<n>Existing unsupervised anomaly detection methods face fundamental challenges including conflicting distributional assumptions, computational inefficiency, and difficulty handling different anomaly types.<n>We propose ISER (Isolation-based Spherical Ensemble Representations) that extends existing isolation-based methods by using hypersphere radii as proxies for local density characteristics while maintaining linear time and constant space complexity.
  arXiv  Detail & Related papers  (2025-10-15T09:00:05Z)
• Towards Generalizable Context-aware Anomaly Detection: A Large-scale Benchmark in Cloud Environments [9.465361225779738]
  Anomaly detection in cloud environments remains both critical and challenging.<n>Existing context-level benchmarks typically focus on either metrics or logs.<n>We introduce CloudAnoBench, a large-scale benchmark for context anomalies in cloud environments.
  arXiv  Detail & Related papers  (2025-08-03T16:59:43Z)
• Automatically Identify and Rectify: Robust Deep Contrastive Multi-view Clustering in Noisy Scenarios [76.02688769599686]
  We propose a novel multi-view clustering framework for the automatic identification and rectification of noisy data, termed AIRMVC.<n>Specifically, we reformulate noisy identification as an anomaly identification problem using GMM.<n>We then design a hybrid rectification strategy to mitigate the adverse effects of noisy data based on the identification results.
  arXiv  Detail & Related papers  (2025-05-27T16:16:54Z)
• GAL-MAD: Towards Explainable Anomaly Detection in Microservice Applications Using Graph Attention Networks [1.0136215038345013]
  Anomalies stemming from network and performance issues must be swiftly identified and addressed. Existing anomaly detection techniques often rely on statistical models or machine learning methods. We propose a novel anomaly detection model called Graph Attention and LSTM-based Microservice Anomaly Detection (GAL-MAD)
  arXiv  Detail & Related papers  (2025-03-31T10:11:31Z)
• LLM Assisted Anomaly Detection Service for Site Reliability Engineers: Enhancing Cloud Infrastructure Resilience [5.644170923282226]
  This paper introduces a scalable Anomaly Detection Service with a generalizable API tailored for industrial time-series data. We provide insights into the usage patterns of the service, with over 500 users and 200,000 API calls in a year. We plan to extend the system to include time series foundation models, enabling zero-shot anomaly detection capabilities.
  arXiv  Detail & Related papers  (2025-01-28T06:41:37Z)
• Anomaly Detection in Large-Scale Cloud Systems: An Industry Case and Dataset [1.293050392312921]
  We introduce a new high-dimensional dataset from IBM Cloud, collected over 4.5 months from the IBM Cloud Console. This dataset comprises 39,365 rows and 117,448 columns of telemetry data. We demonstrate the application of machine learning models for anomaly detection and discuss the key challenges faced in this process.
  arXiv  Detail & Related papers  (2024-11-13T22:04:19Z)
• System States Forecasting of Microservices with Dynamic Spatio-Temporal Data [9.519440926598524]
  Current forecasting methods are insufficient in environments where relationships are critical. In both short-term and long-term forecasting tasks, our model consistently achieved a 8.6% reduction in MAE(Mean Absolute Error) and a 2.2% reduction in MSE (Mean Squared Error)
  arXiv  Detail & Related papers  (2024-08-15T02:52:02Z)
• Root Cause Localization for Microservice Systems in Cloud-edge Collaborative Environments [9.694588952789257]
  Microservice-based software systems face challenges in accurately localizing root causes when failures occur. We propose MicroCERCL, a novel approach that pinpoints root causes at the kernel and application level in the cloud-edge collaborative environment. We show that MicroCERCL can accurately localize the root cause of microservice systems in such environments, significantly outperforming state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-06-19T14:49:37Z)
• PeFAD: A Parameter-Efficient Federated Framework for Time Series Anomaly Detection [51.20479454379662]
  We propose a. Federated Anomaly Detection framework named PeFAD with the increasing privacy concerns. We conduct extensive evaluations on four real datasets, where PeFAD outperforms existing state-of-the-art baselines by up to 28.74%.
  arXiv  Detail & Related papers  (2024-06-04T13:51:08Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• Real-Time Outlier Detection with Dynamic Process Limits [0.609170287691728]
  This paper proposes an online anomaly detection algorithm for existing real-time infrastructures. Online inverse cumulative distribution-based approach is introduced to eliminate common problems of offline anomaly detectors. The benefit of the proposed method is the ease of use, fast computation, and deployability as shown in two case studies of real microgrid operation data.
  arXiv  Detail & Related papers  (2023-01-31T10:23:02Z)
• An Outlier Exposure Approach to Improve Visual Anomaly Detection Performance for Mobile Robots [76.36017224414523]
  We consider the problem of building visual anomaly detection systems for mobile robots. Standard anomaly detection models are trained using large datasets composed only of non-anomalous data. We tackle the problem of exploiting these data to improve the performance of a Real-NVP anomaly detection model.
  arXiv  Detail & Related papers  (2022-09-20T15:18:13Z)
• Toward Certified Robustness Against Real-World Distribution Shifts [65.66374339500025]
  We train a generative model to learn perturbations from data and define specifications with respect to the output of the learned model. A unique challenge arising from this setting is that existing verifiers cannot tightly approximate sigmoid activations. We propose a general meta-algorithm for handling sigmoid activations which leverages classical notions of counter-example-guided abstraction refinement.
  arXiv  Detail & Related papers  (2022-06-08T04:09:13Z)
• Super-resolution GANs of randomly-seeded fields [68.8204255655161]
  We propose a novel super-resolution generative adversarial network (GAN) framework to estimate field quantities from random sparse sensors. The algorithm exploits random sampling to provide incomplete views of the high-resolution underlying distributions. The proposed technique is tested on synthetic databases of fluid flow simulations, ocean surface temperature distributions measurements, and particle image velocimetry data.
  arXiv  Detail & Related papers  (2022-02-23T18:57:53Z)
• TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks [73.01104041298031]
  TadGAN is an unsupervised anomaly detection approach built on Generative Adversarial Networks (GANs) To capture the temporal correlations of time series, we use LSTM Recurrent Neural Networks as base models for Generators and Critics. To demonstrate the performance and generalizability of our approach, we test several anomaly scoring techniques and report the best-suited one.
  arXiv  Detail & Related papers  (2020-09-16T15:52:04Z)
• RobustTAD: Robust Time Series Anomaly Detection via Decomposition and Convolutional Neural Networks [37.16594704493679]
  We propose RobustTAD, a Robust Time series Anomaly Detection framework. It integrates robust seasonal-trend decomposition and convolutional neural network for time series data. It is deployed as a public online service and widely adopted in different business scenarios at Alibaba Group.
  arXiv  Detail & Related papers  (2020-02-21T20:43:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.