How to find a unicorn: a novel model-free, unsupervised anomaly detection method for time series

• URL: http://arxiv.org/abs/2004.11468v3
• Date: Tue, 15 Jun 2021 09:08:02 GMT
• Title: How to find a unicorn: a novel model-free, unsupervised anomaly detection method for time series
• Authors: Zsigmond Benk\H{o}, Tam\'as B\'abel, Zolt\'an Somogyv\'ari
• Abstract summary: We introduce a new anomaly concept called "unicorn" or unique event and present a new, model-free, unsupervised detection algorithm to detect unicorns. The performance of our algorithm was examined in recognizing unique events on different types of simulated data sets with anomalies.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recognition of anomalous events is a challenging but critical task in many scientific and industrial fields, especially when the properties of anomalies are unknown. In this paper, we introduce a new anomaly concept called "unicorn" or unique event and present a new, model-free, unsupervised detection algorithm to detect unicorns. The key component of the new algorithm is the Temporal Outlier Factor (TOF) to measure the uniqueness of events in continuous data sets from dynamic systems. The concept of unique events differs significantly from traditional outliers in many aspects: while repetitive outliers are no longer unique events, a unique event is not necessarily an outlier; it does not necessarily fall out from the distribution of normal activity. The performance of our algorithm was examined in recognizing unique events on different types of simulated data sets with anomalies and it was compared with the Local Outlier Factor (LOF) and discord discovery algorithms. TOF had superior performance compared to LOF and discord algorithms even in recognizing traditional outliers and it also recognized unique events that those did not. The benefits of the unicorn concept and the new detection method were illustrated by example data sets from very different scientific fields. Our algorithm successfully recognized unique events in those cases where they were already known such as the gravitational waves of a binary black hole merger on LIGO detector data and the signs of respiratory failure on ECG data series. Furthermore, unique events were found on the LIBOR data set of the last 30 years.

Related papers

• CaPulse: Detecting Anomalies by Tuning in to the Causal Rhythms of Time Series [42.46322016695622]
  We introduce a new causality-based framework, CaPulse, which tunes in to the underlying causal pulse of time series data to effectively detect anomalies.<n>To tackle the challenges posed by the data, we propose Periodical Normalizing Flows with a novel mask mechanism.<n>Experiments on seven real-world datasets demonstrate that CaPulse consistently outperforms existing methods.
  arXiv  Detail & Related papers  (2025-08-06T16:57:06Z)
• FUN-AD: Fully Unsupervised Learning for Anomaly Detection with Noisy Training Data [1.0650780147044159]
  We propose a novel learning-based approach for fully unsupervised anomaly detection with unlabeled and potentially contaminated training data. Our method is motivated by two observations, that i) the pairwise feature distances between the normal samples are on average likely to be smaller than those between the anomaly samples or heterogeneous samples and ii) pairs of features mutually closest to each other are likely to be homogeneous pairs. Building on the first observation that nearest-neighbor distances can distinguish between confident normal samples and anomalies, we propose a pseudo-labeling strategy using an iteratively reconstructed memory bank.
  arXiv  Detail & Related papers  (2024-11-25T05:51:38Z)
• Open-Set Multivariate Time-Series Anomaly Detection [7.127829790714167]
  Time-series anomaly detection methods assume that only normal samples are available during the training phase. Supervised methods can be utilized to classify normal and seen anomalies, but they tend to overfit to the seen anomalies during training. We propose the first algorithm to address the open-set TSAD problem, called Multivariate Open-Set Time-Series Anomaly Detector (MOSAD) MOSAD is a novel multi-head TSAD framework with a shared representation space and specialized heads, including the Generative head, the Discriminative head, and the Anomaly-Aware Contrastive head.
  arXiv  Detail & Related papers  (2023-10-18T19:55:11Z)
• Abnormal Event Detection via Hypergraph Contrastive Learning [54.80429341415227]
  Abnormal event detection plays an important role in many real applications. In this paper, we study the unsupervised abnormal event detection problem in Attributed Heterogeneous Information Network. A novel hypergraph contrastive learning method, named AEHCL, is proposed to fully capture abnormal event patterns.
  arXiv  Detail & Related papers  (2023-04-02T08:23:20Z)
• Are we certain it's anomalous? [57.729669157989235]
  Anomaly detection in time series is a complex task since anomalies are rare due to highly non-linear temporal correlations. Here we propose the novel use of Hyperbolic uncertainty for Anomaly Detection (HypAD) HypAD learns self-supervisedly to reconstruct the input signal.
  arXiv  Detail & Related papers  (2022-11-16T21:31:39Z)
• Anomaly Rule Detection in Sequence Data [2.3757190901941736]
  We present a new anomaly detection framework called DUOS that enables Discovery of Utility-aware Outlier Sequential rules from a set of sequences. In this work, we incorporate both the anomalousness and utility of a group, and then introduce the concept of utility-aware outlier rule (UOSR)
  arXiv  Detail & Related papers  (2021-11-29T23:52:31Z)
• New Methods and Datasets for Group Anomaly Detection From Fundamental Physics [0.4297070083645048]
  Unsupervised group anomaly detection has become a new frontier of fundamental physics. We propose a realistic synthetic benchmark dataset (LHCO 2020) for the development of group anomaly detection algorithms.
  arXiv  Detail & Related papers  (2021-07-06T18:00:57Z)
• Recomposition vs. Prediction: A Novel Anomaly Detection for Discrete Events Based On Autoencoder [5.781280693720236]
  One of the most challenging problems in the field of intrusion detection is anomaly detection for discrete event logs. We propose DabLog, a Deep Autoencoder-Based anomaly detection method for discrete event Logs. Our approach determines whether a sequence is normal or abnormal by analyzing (encoding) and reconstructing (decoding) the given sequence.
  arXiv  Detail & Related papers  (2020-12-27T16:31:05Z)
• TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks [73.01104041298031]
  TadGAN is an unsupervised anomaly detection approach built on Generative Adversarial Networks (GANs) To capture the temporal correlations of time series, we use LSTM Recurrent Neural Networks as base models for Generators and Critics. To demonstrate the performance and generalizability of our approach, we test several anomaly scoring techniques and report the best-suited one.
  arXiv  Detail & Related papers  (2020-09-16T15:52:04Z)
• Toward Deep Supervised Anomaly Detection: Reinforcement Learning from Partially Labeled Anomaly Data [150.9270911031327]
  We consider the problem of anomaly detection with a small set of partially labeled anomaly examples and a large-scale unlabeled dataset. Existing related methods either exclusively fit the limited anomaly examples that typically do not span the entire set of anomalies, or proceed with unsupervised learning from the unlabeled data. We propose here instead a deep reinforcement learning-based approach that enables an end-to-end optimization of the detection of both labeled and unlabeled anomalies.
  arXiv  Detail & Related papers  (2020-09-15T03:05:39Z)
• Multi-Scale One-Class Recurrent Neural Networks for Discrete Event Sequence Anomaly Detection [63.825781848587376]
  We propose OC4Seq, a one-class recurrent neural network for detecting anomalies in discrete event sequences. Specifically, OC4Seq embeds the discrete event sequences into latent spaces, where anomalies can be easily detected.
  arXiv  Detail & Related papers  (2020-08-31T04:48:22Z)
• Deep Weakly-supervised Anomaly Detection [118.55172352231381]
  Pairwise Relation prediction Network (PReNet) learns pairwise relation features and anomaly scores. PReNet can detect any seen/unseen abnormalities that fit the learned pairwise abnormal patterns. Empirical results on 12 real-world datasets show that PReNet significantly outperforms nine competing methods in detecting seen and unseen anomalies.
  arXiv  Detail & Related papers  (2019-10-30T00:40:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.