Temporal Divide-and-Conquer Anomaly Actions Localization in Semi-Supervised Videos with Hierarchical Transformer

• URL: http://arxiv.org/abs/2408.13643v1
• Date: Sat, 24 Aug 2024 18:12:58 GMT
• Title: Temporal Divide-and-Conquer Anomaly Actions Localization in Semi-Supervised Videos with Hierarchical Transformer
• Authors: Nada Osman, Marwan Torki,
• Abstract summary: Anomaly action detection and localization play an essential role in security and advanced surveillance systems. We propose a hierarchical transformer model designed to evaluate the significance of observed actions in anomalous videos. Our approach segments a parent video hierarchically into multiple temporal children instances and measures the influence of the children nodes in classifying the abnormality of the parent video.
• Score: 0.9208007322096532
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Anomaly action detection and localization play an essential role in security and advanced surveillance systems. However, due to the tremendous amount of surveillance videos, most of the available data for the task is unlabeled or semi-labeled with the video class known, but the location of the anomaly event is unknown. In this work, we target anomaly localization in semi-supervised videos. While the mainstream direction in addressing this task is focused on segment-level multi-instance learning and the generation of pseudo labels, we aim to explore a promising yet unfulfilled direction to solve the problem by learning the temporal relations within videos in order to locate anomaly events. To this end, we propose a hierarchical transformer model designed to evaluate the significance of observed actions in anomalous videos with a divide-and-conquer strategy along the temporal axis. Our approach segments a parent video hierarchically into multiple temporal children instances and measures the influence of the children nodes in classifying the abnormality of the parent video. Evaluating our model on two well-known anomaly detection datasets, UCF-crime and ShanghaiTech, proves its ability to interpret the observed actions within videos and localize the anomalous ones. Our proposed approach outperforms previous works relying on segment-level multiple-instance learning approaches while reaching a promising performance compared to the more recent pseudo-labeling-based approaches.

Related papers

• Weakly Supervised Video Anomaly Detection and Localization with Spatio-Temporal Prompts [57.01985221057047]
  This paper introduces a novel method that learnstemporal prompt embeddings for weakly supervised video anomaly detection and localization (WSVADL) based on pre-trained vision-language models (VLMs) Our method achieves state-of-theart performance on three public benchmarks for the WSVADL task.
  arXiv  Detail & Related papers  (2024-08-12T03:31:29Z)
• Dynamic Erasing Network Based on Multi-Scale Temporal Features for Weakly Supervised Video Anomaly Detection [103.92970668001277]
  We propose a Dynamic Erasing Network (DE-Net) for weakly supervised video anomaly detection. We first propose a multi-scale temporal modeling module, capable of extracting features from segments of varying lengths. Then, we design a dynamic erasing strategy, which dynamically assesses the completeness of the detected anomalies.
  arXiv  Detail & Related papers  (2023-12-04T09:40:11Z)
• Open-Vocabulary Video Anomaly Detection [57.552523669351636]
  Video anomaly detection (VAD) with weak supervision has achieved remarkable performance in utilizing video-level labels to discriminate whether a video frame is normal or abnormal. Recent studies attempt to tackle a more realistic setting, open-set VAD, which aims to detect unseen anomalies given seen anomalies and normal videos. This paper takes a step further and explores open-vocabulary video anomaly detection (OVVAD), in which we aim to leverage pre-trained large models to detect and categorize seen and unseen anomalies.
  arXiv  Detail & Related papers  (2023-11-13T02:54:17Z)
• Holistic Representation Learning for Multitask Trajectory Anomaly Detection [65.72942351514956]
  We propose a holistic representation of skeleton trajectories to learn expected motions across segments at different times. We encode temporally occluded trajectories, jointly learn latent representations of the occluded segments, and reconstruct trajectories based on expected motions across different temporal segments.
  arXiv  Detail & Related papers  (2023-11-03T11:32:53Z)
• Delving into CLIP latent space for Video Anomaly Recognition [24.37974279994544]
  We introduce the novel method AnomalyCLIP, the first to combine Large Language and Vision (LLV) models, such as CLIP. Our approach specifically involves manipulating the latent CLIP feature space to identify the normal event subspace. When anomalous frames are projected onto these directions, they exhibit a large feature magnitude if they belong to a particular class.
  arXiv  Detail & Related papers  (2023-10-04T14:01:55Z)
• Towards Video Anomaly Retrieval from Video Anomaly Detection: New Benchmarks and Model [70.97446870672069]
  Video anomaly detection (VAD) has been paid increasing attention due to its potential applications. Video Anomaly Retrieval ( VAR) aims to pragmatically retrieve relevant anomalous videos by cross-modalities. We present two benchmarks, UCFCrime-AR and XD-Violence, constructed on top of prevalent anomaly datasets.
  arXiv  Detail & Related papers  (2023-07-24T06:22:37Z)
• Anomaly detection in surveillance videos using transformer based attention model [3.2968779106235586]
  This research suggests using a weakly supervised strategy to avoid annotating anomalous segments in training videos. The proposed framework is validated on real-world dataset i.e. ShanghaiTech Campus dataset.
  arXiv  Detail & Related papers  (2022-06-03T12:19:39Z)
• Anomaly Crossing: A New Method for Video Anomaly Detection as Cross-domain Few-shot Learning [32.0713939637202]
  Video anomaly detection aims to identify abnormal events that occurred in videos. Most previous approaches learn only from normal videos using unsupervised or semi-supervised methods. We propose a new learning paradigm by making full use of both normal and abnormal videos for video anomaly detection.
  arXiv  Detail & Related papers  (2021-12-12T20:49:38Z)
• Robust Unsupervised Video Anomaly Detection by Multi-Path Frame Prediction [61.17654438176999]
  We propose a novel and robust unsupervised video anomaly detection method by frame prediction with proper design. Our proposed method obtains the frame-level AUROC score of 88.3% on the CUHK Avenue dataset.
  arXiv  Detail & Related papers  (2020-11-05T11:34:12Z)
• A Self-Reasoning Framework for Anomaly Detection Using Video-Level Labels [17.615297975503648]
  Alous event detection in surveillance videos is a challenging and practical research problem among image and video processing community. We propose a weakly supervised anomaly detection framework based on deep neural networks which is trained in a self-reasoning fashion using only video-level labels. The proposed framework has been evaluated on publicly available real-world anomaly detection datasets including UCF-crime, ShanghaiTech and Ped2.
  arXiv  Detail & Related papers  (2020-08-27T02:14:15Z)
• Localizing Anomalies from Weakly-Labeled Videos [45.58643708315132]
  We propose a WeaklySupervised Anomaly localization (WSAL) method focusing on temporally localizing anomalous segments within anomalous videos. Inspired by the appearance difference in anomalous videos, the evolution of adjacent temporal segments is evaluated for the localization of anomalous segments. Our proposed method achieves new state-of-the-art performance on the UCF-Crime and TAD datasets.
  arXiv  Detail & Related papers  (2020-08-20T12:58:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.