Maat: Performance Metric Anomaly Anticipation for Cloud Services with
Conditional Diffusion
- URL: http://arxiv.org/abs/2308.07676v1
- Date: Tue, 15 Aug 2023 09:50:15 GMT
- Title: Maat: Performance Metric Anomaly Anticipation for Cloud Services with
Conditional Diffusion
- Authors: Cheryl Lee, Tianyi Yang, Zhuangbin Chen, Yuxin Su, Michael R. Lyu
- Abstract summary: Existing techniques for anomaly detection focus solely on real-time detection, meaning that anomaly alerts are issued as soon as anomalies occur.
This paper proposes Maat, the first work to address anomaly anticipation of performance metrics in cloud services.
Maat adopts a novel two-stage paradigm for anomaly anticipation, consisting of metric forecasting and anomaly detection on forecasts.
- Score: 32.86745044103766
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Ensuring the reliability and user satisfaction of cloud services necessitates
prompt anomaly detection followed by diagnosis.
Existing techniques for anomaly detection focus solely on real-time
detection, meaning that anomaly alerts are issued as soon as anomalies occur.
However, anomalies can propagate and escalate into failures, making
faster-than-real-time anomaly detection highly desirable for expediting
downstream analysis and intervention.
This paper proposes Maat, the first work to address anomaly anticipation of
performance metrics in cloud services.
Maat adopts a novel two-stage paradigm for anomaly anticipation, consisting
of metric forecasting and anomaly detection on forecasts.
The metric forecasting stage employs a conditional denoising diffusion model
to enable multi-step forecasting in an auto-regressive manner.
The detection stage extracts anomaly-indicating features based on domain
knowledge and applies isolation forest with incremental learning to detect
upcoming anomalies.
Thus, our method can uncover anomalies that better conform to human
expertise.
Evaluation on three publicly available datasets demonstrates that Maat can
anticipate anomalies faster than real-time comparatively or more effectively
compared with state-of-the-art real-time anomaly detectors.
We also present cases highlighting Maat's success in forecasting abnormal
metrics and discovering anomalies.
Related papers
- Anomaly Detection by Context Contrasting [57.695202846009714]
Anomaly Detection focuses on identifying samples that deviate from the norm.
Recent advances in self-supervised learning have shown great promise in this regard.
We propose Con2, which addresses this problem by setting normal training data into distinct contexts.
Our approach achieves state-of-the-art performance on various benchmarks while exhibiting superior performance in a more realistic healthcare setting.
arXiv Detail & Related papers (2024-05-29T07:59:06Z) - Uncertainty-aware Evaluation of Auxiliary Anomalies with the Expected Anomaly Posterior [17.499560292835]
Anomaly detection is the task of identifying examples that do not behave as expected.
Synthetic anomalies may be of poor quality.
No existing methods quantify the quality of auxiliary anomalies.
arXiv Detail & Related papers (2024-05-22T14:43:29Z) - AnomalyDiffusion: Few-Shot Anomaly Image Generation with Diffusion Model [59.08735812631131]
Anomaly inspection plays an important role in industrial manufacture.
Existing anomaly inspection methods are limited in their performance due to insufficient anomaly data.
We propose AnomalyDiffusion, a novel diffusion-based few-shot anomaly generation model.
arXiv Detail & Related papers (2023-12-10T05:13:40Z) - Precursor-of-Anomaly Detection for Irregular Time Series [31.73234935455713]
We present a novel type of anomaly detection, called Precursor-of-Anomaly (PoA) detection.
To solve both problems at the same time, we present a neural controlled differential equation-based neural network and its multi-task learning algorithm.
arXiv Detail & Related papers (2023-06-27T14:10:09Z) - AGAD: Adversarial Generative Anomaly Detection [12.68966318231776]
Anomaly detection suffered from the lack of anomalies due to the diversity of abnormalities and the difficulties of obtaining large-scale anomaly data.
We propose Adversarial Generative Anomaly Detection (AGAD), a self-contrast-based anomaly detection paradigm.
Our method generates pseudo-anomaly data for both supervised and semi-supervised anomaly detection scenarios.
arXiv Detail & Related papers (2023-04-09T10:40:02Z) - Towards Meaningful Anomaly Detection: The Effect of Counterfactual
Explanations on the Investigation of Anomalies in Multivariate Time Series [0.0]
Among the anomalies detected may be events that are rare, e.g., a planned shutdown of a machine, but are not the actual event of interest.
We propose to support this anomaly investigation by providing explanations of anomaly detection.
We conduct a behavioral experiment using records of taxi rides in New York City as a testbed.
arXiv Detail & Related papers (2023-02-07T07:27:26Z) - Are we certain it's anomalous? [57.729669157989235]
Anomaly detection in time series is a complex task since anomalies are rare due to highly non-linear temporal correlations.
Here we propose the novel use of Hyperbolic uncertainty for Anomaly Detection (HypAD)
HypAD learns self-supervisedly to reconstruct the input signal.
arXiv Detail & Related papers (2022-11-16T21:31:39Z) - Catching Both Gray and Black Swans: Open-set Supervised Anomaly
Detection [90.32910087103744]
A few labeled anomaly examples are often available in many real-world applications.
These anomaly examples provide valuable knowledge about the application-specific abnormality.
Those anomalies seen during training often do not illustrate every possible class of anomaly.
This paper tackles open-set supervised anomaly detection.
arXiv Detail & Related papers (2022-03-28T05:21:37Z) - Explainable Deep Few-shot Anomaly Detection with Deviation Networks [123.46611927225963]
We introduce a novel weakly-supervised anomaly detection framework to train detection models.
The proposed approach learns discriminative normality by leveraging the labeled anomalies and a prior probability.
Our model is substantially more sample-efficient and robust, and performs significantly better than state-of-the-art competing methods in both closed-set and open-set settings.
arXiv Detail & Related papers (2021-08-01T14:33:17Z) - Understanding the Effect of Bias in Deep Anomaly Detection [15.83398707988473]
Anomaly detection presents a unique challenge in machine learning, due to the scarcity of labeled anomaly data.
Recent work attempts to mitigate such problems by augmenting training of deep anomaly detection models with additional labeled anomaly samples.
In this paper, we aim to understand the effect of a biased anomaly set on anomaly detection.
arXiv Detail & Related papers (2021-05-16T03:55:02Z) - Deep Weakly-supervised Anomaly Detection [118.55172352231381]
Pairwise Relation prediction Network (PReNet) learns pairwise relation features and anomaly scores.
PReNet can detect any seen/unseen abnormalities that fit the learned pairwise abnormal patterns.
Empirical results on 12 real-world datasets show that PReNet significantly outperforms nine competing methods in detecting seen and unseen anomalies.
arXiv Detail & Related papers (2019-10-30T00:40:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.