Unsupervised Anomaly Detection in Time-series: An Extensive Evaluation and Analysis of State-of-the-art Methods

• URL: http://arxiv.org/abs/2212.03637v3
• Date: Mon, 12 Aug 2024 06:14:14 GMT
• Title: Unsupervised Anomaly Detection in Time-series: An Extensive Evaluation and Analysis of State-of-the-art Methods
• Authors: Nesryne Mejri, Laura Lopez-Fuentes, Kankana Roy, Pavel Chernakov, Enjie Ghorbel, Djamila Aouada,
• Abstract summary: Unsupervised anomaly detection in time-series has been extensively investigated in the literature. This paper proposes an in-depth evaluation study of recent unsupervised anomaly detection techniques in time-series.
• Score: 10.618572317896515
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Unsupervised anomaly detection in time-series has been extensively investigated in the literature. Notwithstanding the relevance of this topic in numerous application fields, a comprehensive and extensive evaluation of recent state-of-the-art techniques taking into account real-world constraints is still needed. Some efforts have been made to compare existing unsupervised time-series anomaly detection methods rigorously. However, only standard performance metrics, namely precision, recall, and F1-score are usually considered. Essential aspects for assessing their practical relevance are therefore neglected. This paper proposes an in-depth evaluation study of recent unsupervised anomaly detection techniques in time-series. Instead of relying solely on standard performance metrics, additional yet informative metrics and protocols are taken into account. In particular, (i) more elaborate performance metrics specifically tailored for time-series are used; (ii) the model size and the model stability are studied; (iii) an analysis of the tested approaches with respect to the anomaly type is provided; and (iv) a clear and unique protocol is followed for all experiments. Overall, this extensive analysis aims to assess the maturity of state-of-the-art time-series anomaly detection, give insights regarding their applicability under real-world setups and provide to the community a more complete evaluation protocol.

Related papers

• Anomaly Prediction: A Novel Approach with Explicit Delay and Horizon [1.8816077341295625]
  This paper introduces a novel approach for time series anomaly prediction, incorporating temporal information directly into the prediction results. Our results demonstrate the efficacy of our approach in providing timely and accurate anomaly predictions, setting a new benchmark for future research in this field.
  arXiv  Detail & Related papers  (2024-08-08T11:22:52Z)
• A Comprehensive Library for Benchmarking Multi-class Visual Anomaly Detection [52.228708947607636]
  This paper introduces a comprehensive visual anomaly detection benchmark, ADer, which is a modular framework for new methods. The benchmark includes multiple datasets from industrial and medical domains, implementing fifteen state-of-the-art methods and nine comprehensive metrics. We objectively reveal the strengths and weaknesses of different methods and provide insights into the challenges and future directions of multi-class visual anomaly detection.
  arXiv  Detail & Related papers  (2024-06-05T13:40:07Z)
• PATE: Proximity-Aware Time series anomaly Evaluation [3.0377067713090633]
  Traditional performance metrics assume iid data and fail to capture the complex temporal dynamics and specific characteristics of time series anomalies. We introduce Proximity-Aware Time series anomaly Evaluation (PATE), a novel evaluation metric that incorporates the temporal relationship between prediction and anomaly intervals. Our experiments with synthetic and real-world datasets show the superiority of PATE in providing more sensible and accurate evaluations.
  arXiv  Detail & Related papers  (2024-05-20T15:06:36Z)
• Rethinking Out-of-Distribution Detection for Reinforcement Learning: Advancing Methods for Evaluation and Detection [3.7384109981836158]
  We study the problem of out-of-distribution (OOD) detection in reinforcement learning (RL) We propose a clarification of terminology for OOD detection in RL, which aligns it with the literature from other machine learning domains. We present new benchmark scenarios for OOD detection, which introduce anomalies with temporal autocorrelation into different components of the agent-environment loop. We find that DEXTER can reliably identify anomalies across benchmark scenarios, exhibiting superior performance compared to both state-of-the-art OOD detectors and high-dimensional changepoint detectors adopted from statistics.
  arXiv  Detail & Related papers  (2024-04-10T15:39:49Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• On Pixel-level Performance Assessment in Anomaly Detection [87.7131059062292]
  Anomaly detection methods have demonstrated remarkable success across various applications. However, assessing their performance, particularly at the pixel-level, presents a complex challenge. In this paper, we dissect the intricacies of this challenge, underscored by visual evidence and statistical analysis.
  arXiv  Detail & Related papers  (2023-10-25T08:02:27Z)
• CARLA: Self-supervised Contrastive Representation Learning for Time Series Anomaly Detection [53.83593870825628]
  One main challenge in time series anomaly detection (TSAD) is the lack of labelled data in many real-life scenarios. Most of the existing anomaly detection methods focus on learning the normal behaviour of unlabelled time series in an unsupervised manner. We introduce a novel end-to-end self-supervised ContrAstive Representation Learning approach for time series anomaly detection.
  arXiv  Detail & Related papers  (2023-08-18T04:45:56Z)
• PULL: Reactive Log Anomaly Detection Based On Iterative PU Learning [58.85063149619348]
  We propose PULL, an iterative log analysis method for reactive anomaly detection based on estimated failure time windows. Our evaluation shows that PULL consistently outperforms ten benchmark baselines across three different datasets.
  arXiv  Detail & Related papers  (2023-01-25T16:34:43Z)
• Deep Learning for Time Series Anomaly Detection: A Survey [53.83593870825628]
  Time series anomaly detection has applications in a wide range of research fields and applications, including manufacturing and healthcare. The large size and complex patterns of time series have led researchers to develop specialised deep learning models for detecting anomalous patterns. This survey focuses on providing structured and comprehensive state-of-the-art time series anomaly detection models through the use of deep learning.
  arXiv  Detail & Related papers  (2022-11-09T22:40:22Z)
• A Comparative Study on Unsupervised Anomaly Detection for Time Series: Experiments and Analysis [28.79393419730138]
  Time series anomaly detection is often essential to enable reliability and safety. Many recent studies target anomaly detection for time series data. We introduce for data, methods, and evaluation strategies. We systematically evaluate and compare state-of-the-art traditional as well as deep learning techniques.
  arXiv  Detail & Related papers  (2022-09-10T10:44:25Z)
• Anomaly Detection in Predictive Maintenance: A New Evaluation Framework for Temporal Unsupervised Anomaly Detection Algorithms [9.316869851584771]
  The research in anomaly detection lacks a unified definition of what represents an anomalous instance. We propose a concept of positive and negative instances to intervals to be able to evaluate unsupervised anomaly detection algorithms. To show the usefulness of this definition, we include a case study of Big Data algorithms with a real-world time-series problem provided by the company ArcelorMittal.
  arXiv  Detail & Related papers  (2021-05-26T20:15:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.