Unsupervised Event Outlier Detection in Continuous Time

• URL: http://arxiv.org/abs/2411.16427v1
• Date: Mon, 25 Nov 2024 14:29:39 GMT
• Title: Unsupervised Event Outlier Detection in Continuous Time
• Authors: Somjit Nath, Yik Chau Lui, Siqi Liu,
• Abstract summary: We develop, to the best our knowledge, the first unsupervised outlier detection approach to detecting abnormal events. We train a 'generator' that corrects outliers in the data with a 'discriminator' that learns to discriminate the corrected data from the real data. The experimental results show that our method can detect event outliers more accurately than the state-of-the-art approaches.
• Score: 4.375463200687156
• License:
• Abstract: Event sequence data record the occurrences of events in continuous time. Event sequence forecasting based on temporal point processes (TPPs) has been extensively studied, but outlier or anomaly detection, especially without any supervision from humans, is still underexplored. In this work, we develop, to the best our knowledge, the first unsupervised outlier detection approach to detecting abnormal events. Our novel unsupervised outlier detection framework is based on ideas from generative adversarial networks (GANs) and reinforcement learning (RL). We train a 'generator' that corrects outliers in the data with a 'discriminator' that learns to discriminate the corrected data from the real data, which may contain outliers. A key insight is that if the generator made a mistake in the correction, it would generate anomalies that are different from the anomalies in the real data, so it serves as data augmentation for the discriminator learning. Different from typical GAN-based outlier detection approaches, our method employs the generator to detect outliers in an online manner. The experimental results show that our method can detect event outliers more accurately than the state-of-the-art approaches.

Related papers

• Anomaly Detection by Context Contrasting [57.695202846009714]
  Anomaly detection focuses on identifying samples that deviate from the norm. Recent advances in self-supervised learning have shown great promise in this regard. We propose Con$$, which learns through context augmentations.
  arXiv  Detail & Related papers  (2024-05-29T07:59:06Z)
• Adaptive Thresholding Heuristic for KPI Anomaly Detection [1.57731592348751]
  A plethora of outlier detectors have been explored in the time series domain, however, in a business sense, not all outliers are anomalies of interest. This article proposes an Adaptive Thresholding Heuristic (ATH) to dynamically adjust the detection threshold based on the local properties of the data distribution and adapt to changes in time series patterns. Experimental results show that ATH is efficient making it scalable for near real time anomaly detection and flexible with forecasters and outlier detectors.
  arXiv  Detail & Related papers  (2023-08-21T06:45:28Z)
• CARLA: Self-supervised Contrastive Representation Learning for Time Series Anomaly Detection [53.83593870825628]
  One main challenge in time series anomaly detection (TSAD) is the lack of labelled data in many real-life scenarios. Most of the existing anomaly detection methods focus on learning the normal behaviour of unlabelled time series in an unsupervised manner. We introduce a novel end-to-end self-supervised ContrAstive Representation Learning approach for time series anomaly detection.
  arXiv  Detail & Related papers  (2023-08-18T04:45:56Z)
• WePaMaDM-Outlier Detection: Weighted Outlier Detection using Pattern Approaches for Mass Data Mining [0.6754597324022876]
  Outlier detection can reveal vital information about system faults, fraudulent activities, and patterns in the data. This article proposed the WePaMaDM-Outlier Detection with distinct mass data mining domain. It also investigates the significance of data modeling in outlier detection techniques in surveillance, fault detection, and trend analysis.
  arXiv  Detail & Related papers  (2023-06-09T07:00:00Z)
• AGAD: Adversarial Generative Anomaly Detection [12.68966318231776]
  Anomaly detection suffered from the lack of anomalies due to the diversity of abnormalities and the difficulties of obtaining large-scale anomaly data. We propose Adversarial Generative Anomaly Detection (AGAD), a self-contrast-based anomaly detection paradigm. Our method generates pseudo-anomaly data for both supervised and semi-supervised anomaly detection scenarios.
  arXiv  Detail & Related papers  (2023-04-09T10:40:02Z)
• PULL: Reactive Log Anomaly Detection Based On Iterative PU Learning [58.85063149619348]
  We propose PULL, an iterative log analysis method for reactive anomaly detection based on estimated failure time windows. Our evaluation shows that PULL consistently outperforms ten benchmark baselines across three different datasets.
  arXiv  Detail & Related papers  (2023-01-25T16:34:43Z)
• DEGAN: Time Series Anomaly Detection using Generative Adversarial Network Discriminators and Density Estimation [0.0]
  We have proposed an unsupervised Generative Adversarial Network (GAN)-based anomaly detection framework, DEGAN. It relies solely on normal time series data as input to train a well-configured discriminator (D) into a standalone anomaly predictor.
  arXiv  Detail & Related papers  (2022-10-05T04:32:12Z)
• Tracking the risk of a deployed model and detecting harmful distribution shifts [105.27463615756733]
  In practice, it may make sense to ignore benign shifts, under which the performance of a deployed model does not degrade substantially. We argue that a sensible method for firing off a warning has to both (a) detect harmful shifts while ignoring benign ones, and (b) allow continuous monitoring of model performance without increasing the false alarm rate.
  arXiv  Detail & Related papers  (2021-10-12T17:21:41Z)
• DAE : Discriminatory Auto-Encoder for multivariate time-series anomaly detection in air transportation [68.8204255655161]
  We propose a novel anomaly detection model called Discriminatory Auto-Encoder (DAE) It uses the baseline of a regular LSTM-based auto-encoder but with several decoders, each getting data of a specific flight phase. Results show that the DAE achieves better results in both accuracy and speed of detection.
  arXiv  Detail & Related papers  (2021-09-08T14:07:55Z)
• Do We Really Need to Learn Representations from In-domain Data for Outlier Detection? [6.445605125467574]
  Methods based on the two-stage framework achieve state-of-the-art performance on this task. We explore the possibility of avoiding the high cost of training a distinct representation for each outlier detection task. In experiments, we demonstrate competitive or better performance on a variety of outlier detection benchmarks compared with previous two-stage methods.
  arXiv  Detail & Related papers  (2021-05-19T17:30:28Z)
• TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks [73.01104041298031]
  TadGAN is an unsupervised anomaly detection approach built on Generative Adversarial Networks (GANs) To capture the temporal correlations of time series, we use LSTM Recurrent Neural Networks as base models for Generators and Critics. To demonstrate the performance and generalizability of our approach, we test several anomaly scoring techniques and report the best-suited one.
  arXiv  Detail & Related papers  (2020-09-16T15:52:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.