Unsupervised Anomaly Detection in Time-series: An Extensive Evaluation
and Analysis of State-of-the-art Methods
- URL: http://arxiv.org/abs/2212.03637v1
- Date: Tue, 6 Dec 2022 15:05:54 GMT
- Title: Unsupervised Anomaly Detection in Time-series: An Extensive Evaluation
and Analysis of State-of-the-art Methods
- Authors: Nesryne Mejri, Laura Lopez-Fuentes, Kankana Roy, Pavel Chernakov,
Enjie Ghorbel and Djamila Aouada
- Abstract summary: Unsupervised anomaly detection in time-series has been extensively investigated in the literature.
Few efforts have been made to compare existing unsupervised time-series anomaly detection methods rigorously.
This paper proposes an original and in-depth evaluation study of recent unsupervised anomaly detection techniques in time-series.
- Score: 8.953720167142304
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Unsupervised anomaly detection in time-series has been extensively
investigated in the literature. Notwithstanding the relevance of this topic in
numerous application fields, a complete and extensive evaluation of recent
state-of-the-art techniques is still missing. Few efforts have been made to
compare existing unsupervised time-series anomaly detection methods rigorously.
However, only standard performance metrics, namely precision, recall, and
F1-score are usually considered. Essential aspects for assessing their
practical relevance are therefore neglected. This paper proposes an original
and in-depth evaluation study of recent unsupervised anomaly detection
techniques in time-series. Instead of relying solely on standard performance
metrics, additional yet informative metrics and protocols are taken into
account. In particular, (1) more elaborate performance metrics specifically
tailored for time-series are used; (2) the model size and the model stability
are studied; (3) an analysis of the tested approaches with respect to the
anomaly type is provided; and (4) a clear and unique protocol is followed for
all experiments. Overall, this extensive analysis aims to assess the maturity
of state-of-the-art time-series anomaly detection, give insights regarding
their applicability under real-world setups and provide to the community a more
complete evaluation protocol.
Related papers
- ADer: A Comprehensive Benchmark for Multi-class Visual Anomaly Detection [52.228708947607636]
This paper proposes a comprehensive visual anomaly detection benchmark, textbftextitADer, which is a modular framework for new anomaly detection methods.
The benchmark includes multiple datasets from industrial and medical domains, implementing fifteen state-of-the-art methods and nine comprehensive metrics.
We objectively reveal the strengths and weaknesses of different methods and provide insights into the challenges and future directions of multi-class visual anomaly detection.
arXiv Detail & Related papers (2024-06-05T13:40:07Z) - Anomaly Detection by Context Contrasting [57.695202846009714]
Anomaly Detection focuses on identifying samples that deviate from the norm.
Recent advances in self-supervised learning have shown great promise in this regard.
We propose Con2, which addresses this problem by setting normal training data into distinct contexts.
Our approach achieves state-of-the-art performance on various benchmarks while exhibiting superior performance in a more realistic healthcare setting.
arXiv Detail & Related papers (2024-05-29T07:59:06Z) - PATE: Proximity-Aware Time series anomaly Evaluation [3.0377067713090633]
Traditional performance metrics assume iid data and fail to capture the complex temporal dynamics and specific characteristics of time series anomalies.
We introduce Proximity-Aware Time series anomaly Evaluation (PATE), a novel evaluation metric that incorporates the temporal relationship between prediction and anomaly intervals.
Our experiments with synthetic and real-world datasets show the superiority of PATE in providing more sensible and accurate evaluations.
arXiv Detail & Related papers (2024-05-20T15:06:36Z) - Rethinking Out-of-Distribution Detection for Reinforcement Learning: Advancing Methods for Evaluation and Detection [3.7384109981836158]
We study the problem of out-of-distribution (OOD) detection in reinforcement learning (RL)
We propose a clarification of terminology for OOD detection in RL, which aligns it with the literature from other machine learning domains.
We present new benchmark scenarios for OOD detection, which introduce anomalies with temporal autocorrelation into different components of the agent-environment loop.
We find that DEXTER can reliably identify anomalies across benchmark scenarios, exhibiting superior performance compared to both state-of-the-art OOD detectors and high-dimensional changepoint detectors adopted from statistics.
arXiv Detail & Related papers (2024-04-10T15:39:49Z) - Graph Spatiotemporal Process for Multivariate Time Series Anomaly
Detection with Missing Values [67.76168547245237]
We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies.
Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
arXiv Detail & Related papers (2024-01-11T10:10:16Z) - On Pixel-level Performance Assessment in Anomaly Detection [87.7131059062292]
Anomaly detection methods have demonstrated remarkable success across various applications.
However, assessing their performance, particularly at the pixel-level, presents a complex challenge.
In this paper, we dissect the intricacies of this challenge, underscored by visual evidence and statistical analysis.
arXiv Detail & Related papers (2023-10-25T08:02:27Z) - CARLA: Self-supervised Contrastive Representation Learning for Time Series Anomaly Detection [53.83593870825628]
One main challenge in time series anomaly detection (TSAD) is the lack of labelled data in many real-life scenarios.
Most of the existing anomaly detection methods focus on learning the normal behaviour of unlabelled time series in an unsupervised manner.
We introduce a novel end-to-end self-supervised ContrAstive Representation Learning approach for time series anomaly detection.
arXiv Detail & Related papers (2023-08-18T04:45:56Z) - PULL: Reactive Log Anomaly Detection Based On Iterative PU Learning [58.85063149619348]
We propose PULL, an iterative log analysis method for reactive anomaly detection based on estimated failure time windows.
Our evaluation shows that PULL consistently outperforms ten benchmark baselines across three different datasets.
arXiv Detail & Related papers (2023-01-25T16:34:43Z) - A Comparative Study on Unsupervised Anomaly Detection for Time Series:
Experiments and Analysis [28.79393419730138]
Time series anomaly detection is often essential to enable reliability and safety.
Many recent studies target anomaly detection for time series data.
We introduce for data, methods, and evaluation strategies.
We systematically evaluate and compare state-of-the-art traditional as well as deep learning techniques.
arXiv Detail & Related papers (2022-09-10T10:44:25Z) - Anomaly Detection in Predictive Maintenance: A New Evaluation Framework
for Temporal Unsupervised Anomaly Detection Algorithms [9.316869851584771]
The research in anomaly detection lacks a unified definition of what represents an anomalous instance.
We propose a concept of positive and negative instances to intervals to be able to evaluate unsupervised anomaly detection algorithms.
To show the usefulness of this definition, we include a case study of Big Data algorithms with a real-world time-series problem provided by the company ArcelorMittal.
arXiv Detail & Related papers (2021-05-26T20:15:40Z) - Anomaly Detection in Univariate Time-series: A Survey on the
State-of-the-Art [0.0]
Anomaly detection for time-series data has been an important research field for a long time.
Recent years an increasing number of machine learning algorithms have been developed to detect anomalies on time-series.
Researchers tried to improve these techniques using (deep) neural networks.
arXiv Detail & Related papers (2020-04-01T13:22:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.