Maat: Performance Metric Anomaly Anticipation for Cloud Services with Conditional Diffusion

• URL: http://arxiv.org/abs/2308.07676v1
• Date: Tue, 15 Aug 2023 09:50:15 GMT
• Title: Maat: Performance Metric Anomaly Anticipation for Cloud Services with Conditional Diffusion
• Authors: Cheryl Lee, Tianyi Yang, Zhuangbin Chen, Yuxin Su, Michael R. Lyu
• Abstract summary: Existing techniques for anomaly detection focus solely on real-time detection, meaning that anomaly alerts are issued as soon as anomalies occur. This paper proposes Maat, the first work to address anomaly anticipation of performance metrics in cloud services. Maat adopts a novel two-stage paradigm for anomaly anticipation, consisting of metric forecasting and anomaly detection on forecasts.
• Score: 32.86745044103766
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Ensuring the reliability and user satisfaction of cloud services necessitates prompt anomaly detection followed by diagnosis. Existing techniques for anomaly detection focus solely on real-time detection, meaning that anomaly alerts are issued as soon as anomalies occur. However, anomalies can propagate and escalate into failures, making faster-than-real-time anomaly detection highly desirable for expediting downstream analysis and intervention. This paper proposes Maat, the first work to address anomaly anticipation of performance metrics in cloud services. Maat adopts a novel two-stage paradigm for anomaly anticipation, consisting of metric forecasting and anomaly detection on forecasts. The metric forecasting stage employs a conditional denoising diffusion model to enable multi-step forecasting in an auto-regressive manner. The detection stage extracts anomaly-indicating features based on domain knowledge and applies isolation forest with incremental learning to detect upcoming anomalies. Thus, our method can uncover anomalies that better conform to human expertise. Evaluation on three publicly available datasets demonstrates that Maat can anticipate anomalies faster than real-time comparatively or more effectively compared with state-of-the-art real-time anomaly detectors. We also present cases highlighting Maat's success in forecasting abnormal metrics and discovering anomalies.

Related papers

• Adversarially Robust Industrial Anomaly Detection Through Diffusion Model [23.97654469255749]
  We propose a simple yet effective adversarially robust anomaly detection method, textitAdvRAD, that allows the diffusion model to act both as an anomaly detector and adversarial purifier. Our proposed method exhibits outstanding (certified) adversarial robustness while also maintaining equally strong anomaly detection performance on par with the state-of-the-art methods on industrial anomaly detection benchmark datasets.
  arXiv  Detail & Related papers  (2024-08-09T03:25:19Z)
• Uncertainty-aware Evaluation of Auxiliary Anomalies with the Expected Anomaly Posterior [17.499560292835]
  Anomaly detection is the task of identifying examples that do not behave as expected. Synthetic anomalies may be of poor quality. No existing methods quantify the quality of auxiliary anomalies.
  arXiv  Detail & Related papers  (2024-05-22T14:43:29Z)
• AnomalyDiffusion: Few-Shot Anomaly Image Generation with Diffusion Model [59.08735812631131]
  Anomaly inspection plays an important role in industrial manufacture. Existing anomaly inspection methods are limited in their performance due to insufficient anomaly data. We propose AnomalyDiffusion, a novel diffusion-based few-shot anomaly generation model.
  arXiv  Detail & Related papers  (2023-12-10T05:13:40Z)
• MSFlow: Multi-Scale Flow-based Framework for Unsupervised Anomaly Detection [124.52227588930543]
  Unsupervised anomaly detection (UAD) attracts a lot of research interest and drives widespread applications. An inconspicuous yet powerful statistics model, the normalizing flows, is appropriate for anomaly detection and localization in an unsupervised fashion. We propose a novel Multi-Scale Flow-based framework dubbed MSFlow composed of asymmetrical parallel flows followed by a fusion flow. Our MSFlow achieves a new state-of-the-art with a detection AUORC score of up to 99.7%, localization AUCROC score of 98.8%, and PRO score of 97.1%.
  arXiv  Detail & Related papers  (2023-08-29T13:38:35Z)
• Precursor-of-Anomaly Detection for Irregular Time Series [31.73234935455713]
  We present a novel type of anomaly detection, called Precursor-of-Anomaly (PoA) detection. To solve both problems at the same time, we present a neural controlled differential equation-based neural network and its multi-task learning algorithm.
  arXiv  Detail & Related papers  (2023-06-27T14:10:09Z)
• AGAD: Adversarial Generative Anomaly Detection [12.68966318231776]
  Anomaly detection suffered from the lack of anomalies due to the diversity of abnormalities and the difficulties of obtaining large-scale anomaly data. We propose Adversarial Generative Anomaly Detection (AGAD), a self-contrast-based anomaly detection paradigm. Our method generates pseudo-anomaly data for both supervised and semi-supervised anomaly detection scenarios.
  arXiv  Detail & Related papers  (2023-04-09T10:40:02Z)
• Are we certain it's anomalous? [57.729669157989235]
  Anomaly detection in time series is a complex task since anomalies are rare due to highly non-linear temporal correlations. Here we propose the novel use of Hyperbolic uncertainty for Anomaly Detection (HypAD) HypAD learns self-supervisedly to reconstruct the input signal.
  arXiv  Detail & Related papers  (2022-11-16T21:31:39Z)
• Catching Both Gray and Black Swans: Open-set Supervised Anomaly Detection [90.32910087103744]
  A few labeled anomaly examples are often available in many real-world applications. These anomaly examples provide valuable knowledge about the application-specific abnormality. Those anomalies seen during training often do not illustrate every possible class of anomaly. This paper tackles open-set supervised anomaly detection.
  arXiv  Detail & Related papers  (2022-03-28T05:21:37Z)
• Explainable Deep Few-shot Anomaly Detection with Deviation Networks [123.46611927225963]
  We introduce a novel weakly-supervised anomaly detection framework to train detection models. The proposed approach learns discriminative normality by leveraging the labeled anomalies and a prior probability. Our model is substantially more sample-efficient and robust, and performs significantly better than state-of-the-art competing methods in both closed-set and open-set settings.
  arXiv  Detail & Related papers  (2021-08-01T14:33:17Z)
• Understanding the Effect of Bias in Deep Anomaly Detection [15.83398707988473]
  Anomaly detection presents a unique challenge in machine learning, due to the scarcity of labeled anomaly data. Recent work attempts to mitigate such problems by augmenting training of deep anomaly detection models with additional labeled anomaly samples. In this paper, we aim to understand the effect of a biased anomaly set on anomaly detection.
  arXiv  Detail & Related papers  (2021-05-16T03:55:02Z)
• Deep Weakly-supervised Anomaly Detection [118.55172352231381]
  Pairwise Relation prediction Network (PReNet) learns pairwise relation features and anomaly scores. PReNet can detect any seen/unseen abnormalities that fit the learned pairwise abnormal patterns. Empirical results on 12 real-world datasets show that PReNet significantly outperforms nine competing methods in detecting seen and unseen anomalies.
  arXiv  Detail & Related papers  (2019-10-30T00:40:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.