Recomposition vs. Prediction: A Novel Anomaly Detection for Discrete Events Based On Autoencoder

• URL: http://arxiv.org/abs/2012.13972v1
• Date: Sun, 27 Dec 2020 16:31:05 GMT
• Title: Recomposition vs. Prediction: A Novel Anomaly Detection for Discrete Events Based On Autoencoder
• Authors: Lun-Pin Yuan, Peng Liu, Sencun Zhu
• Abstract summary: One of the most challenging problems in the field of intrusion detection is anomaly detection for discrete event logs. We propose DabLog, a Deep Autoencoder-Based anomaly detection method for discrete event Logs. Our approach determines whether a sequence is normal or abnormal by analyzing (encoding) and reconstructing (decoding) the given sequence.
• Score: 5.781280693720236
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: One of the most challenging problems in the field of intrusion detection is anomaly detection for discrete event logs. While most earlier work focused on applying unsupervised learning upon engineered features, most recent work has started to resolve this challenge by applying deep learning methodology to abstraction of discrete event entries. Inspired by natural language processing, LSTM-based anomaly detection models were proposed. They try to predict upcoming events, and raise an anomaly alert when a prediction fails to meet a certain criterion. However, such a predict-next-event methodology has a fundamental limitation: event predictions may not be able to fully exploit the distinctive characteristics of sequences. This limitation leads to high false positives (FPs) and high false negatives (FNs). It is also critical to examine the structure of sequences and the bi-directional causality among individual events. To this end, we propose a new methodology: Recomposing event sequences as anomaly detection. We propose DabLog, a Deep Autoencoder-Based anomaly detection method for discrete event Logs. The fundamental difference is that, rather than predicting upcoming events, our approach determines whether a sequence is normal or abnormal by analyzing (encoding) and reconstructing (decoding) the given sequence. Our evaluation results show that our new methodology can significantly reduce the numbers of FPs and FNs, hence achieving a higher $F_1$ score.

Related papers

• xSemAD: Explainable Semantic Anomaly Detection in Event Logs Using Sequence-to-Sequence Models [1.6713531923053913]
  This work addresses a gap in semantic anomaly detection, which typically indicates the occurrence of an anomaly without explaining the nature of the anomaly. We propose xSemAD, an approach that uses a sequence-to-sequence model to go beyond pure identification and provides extended explanations. Our experiments demonstrate that our approach outperforms existing state-of-the-art semantic anomaly detection methods.
  arXiv  Detail & Related papers  (2024-06-28T09:06:52Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• Abnormal Event Detection via Hypergraph Contrastive Learning [54.80429341415227]
  Abnormal event detection plays an important role in many real applications. In this paper, we study the unsupervised abnormal event detection problem in Attributed Heterogeneous Information Network. A novel hypergraph contrastive learning method, named AEHCL, is proposed to fully capture abnormal event patterns.
  arXiv  Detail & Related papers  (2023-04-02T08:23:20Z)
• Are we certain it's anomalous? [57.729669157989235]
  Anomaly detection in time series is a complex task since anomalies are rare due to highly non-linear temporal correlations. Here we propose the novel use of Hyperbolic uncertainty for Anomaly Detection (HypAD) HypAD learns self-supervisedly to reconstruct the input signal.
  arXiv  Detail & Related papers  (2022-11-16T21:31:39Z)
• Towards Out-of-Distribution Sequential Event Prediction: A Causal Treatment [72.50906475214457]
  The goal of sequential event prediction is to estimate the next event based on a sequence of historical events. In practice, the next-event prediction models are trained with sequential data collected at one time. We propose a framework with hierarchical branching structures for learning context-specific representations.
  arXiv  Detail & Related papers  (2022-10-24T07:54:13Z)
• The Analysis of Online Event Streams: Predicting the Next Activity for Anomaly Detection [0.696125353550498]
  We propose to tackle the online event anomaly detection problem using next-activity prediction methods. We compare these predictive anomaly detection methods to four classical unsupervised anomaly detection approaches.
  arXiv  Detail & Related papers  (2022-03-17T21:17:19Z)
• Anomaly Rule Detection in Sequence Data [2.3757190901941736]
  We present a new anomaly detection framework called DUOS that enables Discovery of Utility-aware Outlier Sequential rules from a set of sequences. In this work, we incorporate both the anomalousness and utility of a group, and then introduce the concept of utility-aware outlier rule (UOSR)
  arXiv  Detail & Related papers  (2021-11-29T23:52:31Z)
• Multi-Scale One-Class Recurrent Neural Networks for Discrete Event Sequence Anomaly Detection [63.825781848587376]
  We propose OC4Seq, a one-class recurrent neural network for detecting anomalies in discrete event sequences. Specifically, OC4Seq embeds the discrete event sequences into latent spaces, where anomalies can be easily detected.
  arXiv  Detail & Related papers  (2020-08-31T04:48:22Z)
• A Background-Agnostic Framework with Adversarial Training for Abnormal Event Detection in Video [120.18562044084678]
  Abnormal event detection in video is a complex computer vision problem that has attracted significant attention in recent years. We propose a background-agnostic framework that learns from training videos containing only normal events.
  arXiv  Detail & Related papers  (2020-08-27T18:39:24Z)
• Sequential Adversarial Anomaly Detection for One-Class Event Data [18.577418448786634]
  We consider the sequential anomaly detection problem in the one-class setting when only the anomalous sequences are available. We propose an adversarial sequential detector by solving a minimax problem to find an optimal detector against the worst-case sequences from a generator.
  arXiv  Detail & Related papers  (2019-10-21T06:12:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.