An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series

• URL: http://arxiv.org/abs/2109.11428v1
• Date: Thu, 23 Sep 2021 15:14:24 GMT
• Title: An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series
• Authors: Astha Garg, Wenyu Zhang, Jules Samaran, Savitha Ramasamy and Chuan-Sheng Foo
• Abstract summary: This paper presents a systematic and comprehensive evaluation of unsupervised and semi-supervised deep-learning based methods for anomaly detection and diagnosis. We vary the model and post-processing of model errors, through a grid of 10 models and 4 scoring functions, comparing these variants to state of the art methods. We find that the existing evaluation metrics either do not take events into account, or cannot distinguish between a good detector and trivial detectors.
• Score: 7.675917669905486
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Several techniques for multivariate time series anomaly detection have been proposed recently, but a systematic comparison on a common set of datasets and metrics is lacking. This paper presents a systematic and comprehensive evaluation of unsupervised and semi-supervised deep-learning based methods for anomaly detection and diagnosis on multivariate time series data from cyberphysical systems. Unlike previous works, we vary the model and post-processing of model errors, i.e. the scoring functions independently of each other, through a grid of 10 models and 4 scoring functions, comparing these variants to state of the art methods. In time-series anomaly detection, detecting anomalous events is more important than detecting individual anomalous time-points. Through experiments, we find that the existing evaluation metrics either do not take events into account, or cannot distinguish between a good detector and trivial detectors, such as a random or an all-positive detector. We propose a new metric to overcome these drawbacks, namely, the composite F-score ($Fc_1$), for evaluating time-series anomaly detection. Our study highlights that dynamic scoring functions work much better than static ones for multivariate time series anomaly detection, and the choice of scoring functions often matters more than the choice of the underlying model. We also find that a simple, channel-wise model - the Univariate Fully-Connected Auto-Encoder, with the dynamic Gaussian scoring function emerges as a winning candidate for both anomaly detection and diagnosis, beating state of the art algorithms.

Related papers

• USD: Unsupervised Soft Contrastive Learning for Fault Detection in Multivariate Time Series [6.055410677780381]
  We introduce a combination of data augmentation and soft contrastive learning, specifically designed to capture the multifaceted nature of state behaviors more accurately. This dual strategy significantly boosts the model's ability to distinguish between normal and abnormal states, leading to a marked improvement in fault detection performance across multiple datasets and settings.
  arXiv  Detail & Related papers  (2024-05-25T14:48:04Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• Effective Abnormal Activity Detection on Multivariate Time Series Healthcare Data [8.84352369893021]
  We propose a Residual-based Anomaly Detection approach, Rs-AD, for effective representation learning and abnormal activity detection. We evaluate our scheme on a real-world gait dataset and the experimental results demonstrate an F1 score of 0.839.
  arXiv  Detail & Related papers  (2023-09-11T22:08:09Z)
• MadSGM: Multivariate Anomaly Detection with Score-based Generative Models [22.296610226476542]
  We present a time-series anomaly detector based on score-based generative models, called MadSGM. Experiments on five real-world benchmark datasets illustrate that MadSGM achieves the most robust and accurate predictions.
  arXiv  Detail & Related papers  (2023-08-29T07:04:50Z)
• CARLA: Self-supervised Contrastive Representation Learning for Time Series Anomaly Detection [53.83593870825628]
  One main challenge in time series anomaly detection (TSAD) is the lack of labelled data in many real-life scenarios. Most of the existing anomaly detection methods focus on learning the normal behaviour of unlabelled time series in an unsupervised manner. We introduce a novel end-to-end self-supervised ContrAstive Representation Learning approach for time series anomaly detection.
  arXiv  Detail & Related papers  (2023-08-18T04:45:56Z)
• Deep Learning for Time Series Anomaly Detection: A Survey [53.83593870825628]
  Time series anomaly detection has applications in a wide range of research fields and applications, including manufacturing and healthcare. The large size and complex patterns of time series have led researchers to develop specialised deep learning models for detecting anomalous patterns. This survey focuses on providing structured and comprehensive state-of-the-art time series anomaly detection models through the use of deep learning.
  arXiv  Detail & Related papers  (2022-11-09T22:40:22Z)
• Causality-Based Multivariate Time Series Anomaly Detection [63.799474860969156]
  We formulate the anomaly detection problem from a causal perspective and view anomalies as instances that do not follow the regular causal mechanism to generate the multivariate data. We then propose a causality-based anomaly detection approach, which first learns the causal structure from data and then infers whether an instance is an anomaly relative to the local causal mechanism. We evaluate our approach with both simulated and public datasets as well as a case study on real-world AIOps applications.
  arXiv  Detail & Related papers  (2022-06-30T06:00:13Z)
• HIFI: Anomaly Detection for Multivariate Time Series with High-order Feature Interactions [7.016615391171876]
  HIFI builds multivariate feature interaction graph automatically and uses the graph convolutional neural network to achieve high-order feature interactions. Experiments on three publicly available datasets demonstrate the superiority of our framework compared with state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-06-11T04:57:03Z)
• Consistency of mechanistic causal discovery in continuous-time using Neural ODEs [85.7910042199734]
  We consider causal discovery in continuous-time for the study of dynamical systems. We propose a causal discovery algorithm based on penalized Neural ODEs.
  arXiv  Detail & Related papers  (2021-05-06T08:48:02Z)
• TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks [73.01104041298031]
  TadGAN is an unsupervised anomaly detection approach built on Generative Adversarial Networks (GANs) To capture the temporal correlations of time series, we use LSTM Recurrent Neural Networks as base models for Generators and Critics. To demonstrate the performance and generalizability of our approach, we test several anomaly scoring techniques and report the best-suited one.
  arXiv  Detail & Related papers  (2020-09-16T15:52:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.