MemStream: Memory-Based Anomaly Detection in Multi-Aspect Streams with Concept Drift

• URL: http://arxiv.org/abs/2106.03837v1
• Date: Mon, 7 Jun 2021 17:54:57 GMT
• Title: MemStream: Memory-Based Anomaly Detection in Multi-Aspect Streams with Concept Drift
• Authors: Siddharth Bhatia, Arjit Jain, Shivin Srivastava, Kenji Kawaguchi, Bryan Hooi
• Abstract summary: We propose MemStream, a streaming multi-aspect anomaly detection framework. We leverage the power of a denoising autoencoder to learn representations and a memory module to learn the dynamically changing trend in data. Experimental results show the effectiveness of our approach compared to state-of-the-art streaming baselines.
• Score: 20.143379054091536
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Given a stream of entries over time in a multi-aspect data setting where concept drift is present, how can we detect anomalous activities? Most of the existing unsupervised anomaly detection approaches seek to detect anomalous events in an offline fashion and require a large amount of data for training. This is not practical in real-life scenarios where we receive the data in a streaming manner and do not know the size of the stream beforehand. Thus, we need a data-efficient method that can detect and adapt to changing data trends, or concept drift, in an online manner. In this work, we propose MemStream, a streaming multi-aspect anomaly detection framework, allowing us to detect unusual events as they occur while being resilient to concept drift. We leverage the power of a denoising autoencoder to learn representations and a memory module to learn the dynamically changing trend in data without the need for labels. We prove the optimum memory size required for effective drift handling. Furthermore, MemStream makes use of two architecture design choices to be robust to memory poisoning. Experimental results show the effectiveness of our approach compared to state-of-the-art streaming baselines using 2 synthetic datasets and 11 real-world datasets.

Related papers

• CADM: Confusion Model-based Detection Method for Real-drift in Chunk Data Stream [3.0885191226198785]
  Concept drift detection has attracted considerable attention due to its importance in many real-world applications such as health monitoring and fault diagnosis. We propose a new approach to detect real-drift in the chunk data stream with limited annotations based on concept confusion.
  arXiv  Detail & Related papers  (2023-03-25T08:59:27Z)
• Dual Memory Aggregation Network for Event-Based Object Detection with Learnable Representation [79.02808071245634]
  Event-based cameras are bio-inspired sensors that capture brightness change of every pixel in an asynchronous manner. Event streams are divided into grids in the x-y-t coordinates for both positive and negative polarity, producing a set of pillars as 3D tensor representation. Long memory is encoded in the hidden state of adaptive convLSTMs while short memory is modeled by computing spatial-temporal correlation between event pillars.
  arXiv  Detail & Related papers  (2023-03-17T12:12:41Z)
• Streaming Anomaly Detection [7.60882697435906]
  We first propose MIDAS which uses a count-min sketch to detect anomalous edges in dynamic graphs in an online manner. We then extend the count-min sketch to a Higher-Order sketch to capture complex relations in graph data. Using this sketch, we propose four streaming methods to detect edge and subgraph anomalies.
  arXiv  Detail & Related papers  (2023-01-30T18:59:51Z)
• TempNet: Temporal Attention Towards the Detection of Animal Behaviour in Videos [63.85815474157357]
  We propose an efficient computer vision- and deep learning-based method for the detection of biological behaviours in videos. TempNet uses an encoder bridge and residual blocks to maintain model performance with a two-staged, spatial, then temporal, encoder. We demonstrate its application to the detection of sablefish (Anoplopoma fimbria) startle events.
  arXiv  Detail & Related papers  (2022-11-17T23:55:12Z)
• Context Recovery and Knowledge Retrieval: A Novel Two-Stream Framework for Video Anomaly Detection [48.05512963355003]
  We propose a two-stream framework based on context recovery and knowledge retrieval. For the context recovery stream, we propose a U-Net which can fully utilize the motion information to predict the future frame. For the knowledge retrieval stream, we propose an improved learnable locality-sensitive hashing. The knowledge about normality is encoded and stored in hash tables, and the distance between the testing event and the knowledge representation is used to reveal the probability of anomaly.
  arXiv  Detail & Related papers  (2022-09-07T03:12:02Z)
• StreamYOLO: Real-time Object Detection for Streaming Perception [84.2559631820007]
  We endow the models with the capacity of predicting the future, significantly improving the results for streaming perception. We consider multiple velocities driving scene and propose Velocity-awared streaming AP (VsAP) to jointly evaluate the accuracy. Our simple method achieves the state-of-the-art performance on Argoverse-HD dataset and improves the sAP and VsAP by 4.7% and 8.2% respectively.
  arXiv  Detail & Related papers  (2022-07-21T12:03:02Z)
• Real-time Object Detection for Streaming Perception [84.2559631820007]
  Streaming perception is proposed to jointly evaluate the latency and accuracy into a single metric for video online perception. We build a simple and effective framework for streaming perception. Our method achieves competitive performance on Argoverse-HD dataset and improves the AP by 4.9% compared to the strong baseline.
  arXiv  Detail & Related papers  (2022-03-23T11:33:27Z)
• Detecting Concept Drift With Neural Network Model Uncertainty [0.0]
  Uncertainty Drift Detection (UDD) is able to detect drifts without access to true labels. In contrast to input data-based drift detection, our approach considers the effects of the current input data on the properties of the prediction model. We show that UDD outperforms other state-of-the-art strategies on two synthetic as well as ten real-world data sets for both regression and classification tasks.
  arXiv  Detail & Related papers  (2021-07-05T08:56:36Z)
• Xtreaming: an incremental multidimensional projection technique and its application to streaming data [58.92615359254597]
  Xtreaming is a novel incremental projection technique that continuously updates the visual representation to reflect new emerging structures or patterns without visiting the multidimensional data more than once. Our tests show that Xtreaming is competitive in terms of global distance preservation if compared to other streaming and incremental techniques.
  arXiv  Detail & Related papers  (2020-03-08T04:53:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.