Making the End-User a Priority in Benchmarking: OrionBench for Unsupervised Time Series Anomaly Detection

• URL: http://arxiv.org/abs/2310.17748v2
• Date: Mon, 4 Mar 2024 20:39:19 GMT
• Title: Making the End-User a Priority in Benchmarking: OrionBench for Unsupervised Time Series Anomaly Detection
• Authors: Sarah Alnegheimish, Laure Berti-Equille, Kalyan Veeramachaneni
• Abstract summary: Time series anomaly detection is a prevalent problem in many application domains such as patient monitoring in healthcare, forecasting in finance, or predictive maintenance in energy. We propose OrionBench -- a user centric continuously maintained benchmark for unsupervised time series anomaly detection. We demonstrate the usage of OrionBench, and the progression of pipelines across 16 releases published over the course of three years.
• Score: 9.054540533394924
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Time series anomaly detection is a prevalent problem in many application domains such as patient monitoring in healthcare, forecasting in finance, or predictive maintenance in energy. This has led to the emergence of a plethora of anomaly detection methods, including more recently, deep learning based methods. Although several benchmarks have been proposed to compare newly developed models, they usually rely on one-time execution over a limited set of datasets and the comparison is restricted to a few models. We propose OrionBench -- a user centric continuously maintained benchmark for unsupervised time series anomaly detection. The framework provides universal abstractions to represent models, extensibility to add new pipelines and datasets, hyperparameter standardization, pipeline verification, and frequent releases with published benchmarks. We demonstrate the usage of OrionBench, and the progression of pipelines across 16 releases published over the course of three years. Moreover, we walk through two real scenarios we experienced with OrionBench that highlight the importance of continuous benchmarks in unsupervised time series anomaly detection.

Related papers

• TimeSeriesBench: An Industrial-Grade Benchmark for Time Series Anomaly Detection Models [21.658019069964755]
  Time series anomaly detection (TSAD) has gained significant attention due to its real-world applications. There is no effective way to verify whether TSAD can meet the requirements for real-world deployment. We propose an industrial-grade benchmark TimeSeriesBench to assess the performance of existing algorithms.
  arXiv  Detail & Related papers  (2024-02-16T16:25:20Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• LARA: A Light and Anti-overfitting Retraining Approach for Unsupervised Time Series Anomaly Detection [49.52429991848581]
  We propose a Light and Anti-overfitting Retraining Approach (LARA) for deep variational auto-encoder based time series anomaly detection methods (VAEs) This work aims to make three novel contributions: 1) the retraining process is formulated as a convex problem and can converge at a fast rate as well as prevent overfitting; 2) designing a ruminate block, which leverages the historical data without the need to store them; and 3) mathematically proving that when fine-tuning the latent vector and reconstructed data, the linear formations can achieve the least adjusting errors between the ground truths and the fine-tuned ones.
  arXiv  Detail & Related papers  (2023-10-09T12:36:16Z)
• Low-count Time Series Anomaly Detection [1.3207844222875191]
  Low-count time series describe sparse or intermittent events, which are prevalent in large-scale online platforms that capture and monitor diverse data types. Several distinct challenges surface when modelling low-count time series, particularly low signal-to-noise ratios. We introduce a novel generative procedure for creating benchmark datasets comprising of low-count time series with anomalous segments.
  arXiv  Detail & Related papers  (2023-08-24T16:58:30Z)
• CARLA: Self-supervised Contrastive Representation Learning for Time Series Anomaly Detection [53.83593870825628]
  One main challenge in time series anomaly detection (TSAD) is the lack of labelled data in many real-life scenarios. Most of the existing anomaly detection methods focus on learning the normal behaviour of unlabelled time series in an unsupervised manner. We introduce a novel end-to-end self-supervised ContrAstive Representation Learning approach for time series anomaly detection.
  arXiv  Detail & Related papers  (2023-08-18T04:45:56Z)
• Time series anomaly detection with reconstruction-based state-space models [10.085100442558828]
  We propose a novel unsupervised anomaly detection method for time series data. A long short-term memory (LSTM)-based encoder-decoder is adopted to represent the mapping between the observation space and the latent space. Regularization of the latent space places constraints on the states of normal samples, and Mahalanobis distance is used to evaluate the abnormality level.
  arXiv  Detail & Related papers  (2023-03-06T17:52:35Z)
• Multi-scale Anomaly Detection for Big Time Series of Industrial Sensors [50.6434162489902]
  We propose a reconstruction-based anomaly detection method, MissGAN, iteratively learning to decode and encode naturally smooth time series. MissGAN does not need labels or only needs labels of normal instances, making it widely applicable.
  arXiv  Detail & Related papers  (2022-04-18T04:34:15Z)
• TiSAT: Time Series Anomaly Transformer [30.68108039722565]
  We show that a rudimentary Random Guess method can outperform state-of-the-art detectors in terms of this popular but faulty evaluation criterion. In this work, we propose a proper evaluation metric that measures the timeliness and precision of detecting sequential anomalies.
  arXiv  Detail & Related papers  (2022-03-10T05:46:58Z)
• TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks [73.01104041298031]
  TadGAN is an unsupervised anomaly detection approach built on Generative Adversarial Networks (GANs) To capture the temporal correlations of time series, we use LSTM Recurrent Neural Networks as base models for Generators and Critics. To demonstrate the performance and generalizability of our approach, we test several anomaly scoring techniques and report the best-suited one.
  arXiv  Detail & Related papers  (2020-09-16T15:52:04Z)
• Anomaly Detection at Scale: The Case for Deep Distributional Time Series Models [14.621700495712647]
  Main novelty in our approach is that instead of modeling time series consisting of real values or vectors of real values, we model time series of probability distributions over real values (or vectors) Our method is amenable to streaming anomaly detection and scales to monitoring for anomalies on millions of time series. We show that we outperform popular open-source anomaly detection tools by up to 17% average improvement for a real-world data set.
  arXiv  Detail & Related papers  (2020-07-30T15:48:55Z)
• A Multi-Channel Neural Graphical Event Model with Negative Evidence [76.51278722190607]
  Event datasets are sequences of events of various types occurring irregularly over the time-line. We propose a non-parametric deep neural network approach in order to estimate the underlying intensity functions.
  arXiv  Detail & Related papers  (2020-02-21T23:10:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.