Transformer Based Self-Context Aware Prediction for Few-Shot Anomaly Detection in Videos

• URL: http://arxiv.org/abs/2503.00670v1
• Date: Sun, 02 Mar 2025 00:07:49 GMT
• Title: Transformer Based Self-Context Aware Prediction for Few-Shot Anomaly Detection in Videos
• Authors: Gargi V. Pillai, Ashish Verma, Debashis Sen,
• Abstract summary: We propose a one-class few-shot learning driven transformer based approach for anomaly detection in videos that is self-context aware.<n>Features from the first few consecutive non-anomalous frames in a video are used to train the transformer in predicting the non-anomalous feature of the subsequent frame.<n>After the learning, given a few previous frames, the video-specific transformer is used to infer if a frame is anomalous or not by comparing the feature predicted by it with the actual.
• Score: 8.773238774969068
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Anomaly detection in videos is a challenging task as anomalies in different videos are of different kinds. Therefore, a promising way to approach video anomaly detection is by learning the non-anomalous nature of the video at hand. To this end, we propose a one-class few-shot learning driven transformer based approach for anomaly detection in videos that is self-context aware. Features from the first few consecutive non-anomalous frames in a video are used to train the transformer in predicting the non-anomalous feature of the subsequent frame. This takes place under the attention of a self-context learned from the input features themselves. After the learning, given a few previous frames, the video-specific transformer is used to infer if a frame is anomalous or not by comparing the feature predicted by it with the actual. The effectiveness of the proposed method with respect to the state-of-the-art is demonstrated through qualitative and quantitative results on different standard datasets. We also study the positive effect of the self-context used in our approach.

Related papers

• Anomaly detection in non-stationary videos using time-recursive differencing network based prediction [6.107978190324034]
  We propose to perform prediction using a time-recursive differencing network followed by autoregressive moving average estimation for video anomaly detection.<n>The effectiveness of the proposed approach is demonstrated considering a simple optical flow based video feature.
  arXiv  Detail & Related papers  (2025-03-04T03:16:39Z)
• Patch Spatio-Temporal Relation Prediction for Video Anomaly Detection [19.643936110623653]
  Video Anomaly Detection (VAD) aims to identify abnormalities within a specific context and timeframe. Recent deep learning-based VAD models have shown promising results by generating high-resolution frames. We propose a self-supervised learning approach for VAD through an inter-patch relationship prediction task.
  arXiv  Detail & Related papers  (2024-03-28T03:07:16Z)
• Time Is MattEr: Temporal Self-supervision for Video Transformers [72.42240984211283]
  We design simple yet effective self-supervised tasks for video models to learn temporal dynamics better. Our method learns the temporal order of video frames as extra self-supervision and enforces the randomly shuffled frames to have low-confidence outputs. Under various video action recognition tasks, we demonstrate the effectiveness of our method and its compatibility with state-of-the-art Video Transformers.
  arXiv  Detail & Related papers  (2022-07-19T04:44:08Z)
• Multi-Contextual Predictions with Vision Transformer for Video Anomaly Detection [22.098399083491937]
  understanding of thetemporal context of a video plays a vital role in anomaly detection. We design a transformer model with three different contextual prediction streams: masked, whole and partial. By learning to predict the missing frames of consecutive normal frames, our model can effectively learn various normality patterns in the video.
  arXiv  Detail & Related papers  (2022-06-17T05:54:31Z)
• Anomaly detection in surveillance videos using transformer based attention model [3.2968779106235586]
  This research suggests using a weakly supervised strategy to avoid annotating anomalous segments in training videos. The proposed framework is validated on real-world dataset i.e. ShanghaiTech Campus dataset.
  arXiv  Detail & Related papers  (2022-06-03T12:19:39Z)
• Approaches Toward Physical and General Video Anomaly Detection [0.0]
  Anomaly detection in videos may enable automatic detection of malfunctions in many manufacturing, maintenance, and real-life settings. We introduce the Physical Anomalous Trajectory or Motion dataset, which contains six different video classes. We suggest an even harder benchmark where anomalous activities should be spotted on highly variable scenes.
  arXiv  Detail & Related papers  (2021-12-14T18:57:44Z)
• Scene Uncertainty and the Wellington Posterior of Deterministic Image Classifiers [68.9065881270224]
  We introduce the Wellington Posterior, which is the distribution of outcomes that would have been obtained in response to data that could have been generated by the same scene. We explore the use of data augmentation, dropout, ensembling, single-view reconstruction, and model linearization to compute a Wellington Posterior. Additional methods include the use of conditional generative models such as generative adversarial networks, neural radiance fields, and conditional prior networks.
  arXiv  Detail & Related papers  (2021-06-25T20:10:00Z)
• Robust Unsupervised Video Anomaly Detection by Multi-Path Frame Prediction [61.17654438176999]
  We propose a novel and robust unsupervised video anomaly detection method by frame prediction with proper design. Our proposed method obtains the frame-level AUROC score of 88.3% on the CUHK Avenue dataset.
  arXiv  Detail & Related papers  (2020-11-05T11:34:12Z)
• Blind Video Temporal Consistency via Deep Video Prior [61.062900556483164]
  We present a novel and general approach for blind video temporal consistency. Our method is only trained on a pair of original and processed videos directly. We show that temporal consistency can be achieved by training a convolutional network on a video with the Deep Video Prior.
  arXiv  Detail & Related papers  (2020-10-22T16:19:20Z)
• Uncertainty-Aware Weakly Supervised Action Detection from Untrimmed Videos [82.02074241700728]
  In this paper, we present a prohibitive-level action recognition model that is trained with only video-frame labels. Our method per person detectors have been trained on large image datasets within Multiple Instance Learning framework. We show how we can apply our method in cases where the standard Multiple Instance Learning assumption, that each bag contains at least one instance with the specified label, is invalid.
  arXiv  Detail & Related papers  (2020-07-21T10:45:05Z)
• Non-Adversarial Video Synthesis with Learned Priors [53.26777815740381]
  We focus on the problem of generating videos from latent noise vectors, without any reference input frames. We develop a novel approach that jointly optimize the input latent space, the weights of a recurrent neural network and a generator through non-adversarial learning. Our approach generates superior quality videos compared to the existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-03-21T02:57:33Z)
• Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detection [114.9714355807607]
  We show that applying self-trained deep ordinal regression to video anomaly detection overcomes two key limitations of existing methods. We devise an end-to-end trainable video anomaly detection approach that enables joint representation learning and anomaly scoring without manually labeled normal/abnormal data.
  arXiv  Detail & Related papers  (2020-03-15T08:44:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.