GMFVAD: Using Grained Multi-modal Feature to Improve Video Anomaly Detection

• URL: http://arxiv.org/abs/2510.20268v1
• Date: Thu, 23 Oct 2025 06:52:53 GMT
• Title: GMFVAD: Using Grained Multi-modal Feature to Improve Video Anomaly Detection
• Authors: Guangyu Dai, Dong Chen, Siliang Tang, Yueting Zhuang,
• Abstract summary: We propose Grained Multi-modal Feature for Video Anomaly Detection (GMFVAD)<n>We generate more grained multi-modal feature based on the video snippet, which summarizes the main content.<n> Experiments show that GMFVAD achieves state-of-the-art performance on four mainly datasets.
• Score: 63.16754542429089
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video anomaly detection (VAD) is a challenging task that detects anomalous frames in continuous surveillance videos. Most previous work utilizes the spatio-temporal correlation of visual features to distinguish whether there are abnormalities in video snippets. Recently, some works attempt to introduce multi-modal information, like text feature, to enhance the results of video anomaly detection. However, these works merely incorporate text features into video snippets in a coarse manner, overlooking the significant amount of redundant information that may exist within the video snippets. Therefore, we propose to leverage the diversity among multi-modal information to further refine the extracted features, reducing the redundancy in visual features, and we propose Grained Multi-modal Feature for Video Anomaly Detection (GMFVAD). Specifically, we generate more grained multi-modal feature based on the video snippet, which summarizes the main content, and text features based on the captions of original video will be introduced to further enhance the visual features of highlighted portions. Experiments show that the proposed GMFVAD achieves state-of-the-art performance on four mainly datasets. Ablation experiments also validate that the improvement of GMFVAD is due to the reduction of redundant information.

Related papers

• VideoPerceiver: Enhancing Fine-Grained Temporal Perception in Video Multimodal Large Language Models [9.896951371033229]
  VideoPerceiver is a video multimodal large language model (VMLLM) that enhances fine-grained perception in video understanding.<n>We construct "key-information-missing" videos by extracting event-action keywords from captions, identifying corresponding key frames, and replacing them with adjacent frames.<n> Experiments show VideoPerceiver substantially outperforms state-of-the-art VMLLMs on fine-grained action understanding and rare event captioning benchmarks.
  arXiv  Detail & Related papers  (2025-11-24T06:57:26Z)
• Enhanced Multimodal Hate Video Detection via Channel-wise and Modality-wise Fusion [7.728348842555291]
  The rapid rise of video content on platforms such as TikTok and YouTube has transformed information dissemination.<n>Despite significant efforts to combat hate speech, detecting these videos remains challenging due to their often implicit nature.<n>We present CMFusion, an enhanced multimodal hate video detection model utilizing a novel Channel-wise and Modality-wise Fusion Mechanism.
  arXiv  Detail & Related papers  (2025-05-17T15:24:48Z)
• AVadCLIP: Audio-Visual Collaboration for Robust Video Anomaly Detection [57.649223695021114]
  We present a novel weakly supervised framework that leverages audio-visual collaboration for robust video anomaly detection.<n>Our framework demonstrates superior performance across multiple benchmarks, with audio integration significantly boosting anomaly detection accuracy.
  arXiv  Detail & Related papers  (2025-04-06T13:59:16Z)
• On Learning Multi-Modal Forgery Representation for Diffusion Generated Video Detection [44.55891118519547]
  We propose an innovative algorithm named Multi-Mod-al Detection(MM-Det) for detecting diffusion-generated content.<n>MM-Det utilizes the profound and comprehensive abilities of Large Multi-modal Models (LMMs) by generating a Multi-Modal Forgery Representation (MMFR)<n>MM-Det achieves state-of-the-art performance in Video Forensics (DVF)
  arXiv  Detail & Related papers  (2024-10-31T04:20:47Z)
• EVC-MF: End-to-end Video Captioning Network with Multi-scale Features [13.85795110061781]
  We propose an end-to-end encoder-decoder-based network (EVC-MF) for video captioning. It efficiently utilizes multi-scale visual and textual features to generate video descriptions. The results demonstrate that EVC-MF yields competitive performance compared with the state-of-theart methods.
  arXiv  Detail & Related papers  (2024-10-22T02:16:02Z)
• VANE-Bench: Video Anomaly Evaluation Benchmark for Conversational LMMs [64.60035916955837]
  VANE-Bench is a benchmark designed to assess the proficiency of Video-LMMs in detecting anomalies and inconsistencies in videos.<n>Our dataset comprises an array of videos synthetically generated using existing state-of-the-art text-to-video generation models.<n>We evaluate nine existing Video-LMMs, both open and closed sources, on this benchmarking task and find that most of the models encounter difficulties in effectively identifying the subtle anomalies.
  arXiv  Detail & Related papers  (2024-06-14T17:59:01Z)
• Dynamic Erasing Network Based on Multi-Scale Temporal Features for Weakly Supervised Video Anomaly Detection [103.92970668001277]
  We propose a Dynamic Erasing Network (DE-Net) for weakly supervised video anomaly detection. We first propose a multi-scale temporal modeling module, capable of extracting features from segments of varying lengths. Then, we design a dynamic erasing strategy, which dynamically assesses the completeness of the detected anomalies.
  arXiv  Detail & Related papers  (2023-12-04T09:40:11Z)
• Video Infringement Detection via Feature Disentanglement and Mutual Information Maximization [51.206398602941405]
  We propose to disentangle an original high-dimensional feature into multiple sub-features. On top of the disentangled sub-features, we learn an auxiliary feature to enhance the sub-features. Our method achieves 90.1% TOP-100 mAP on the large-scale SVD dataset and also sets the new state-of-the-art on the VCSL benchmark dataset.
  arXiv  Detail & Related papers  (2023-09-13T10:53:12Z)
• Multimodal Graph Learning for Deepfake Detection [10.077496841634135]
  Existing deepfake detectors face several challenges in achieving robustness and generalization. We propose a novel framework, namely Multimodal Graph Learning (MGL), that leverages information from multiple modalities. Our proposed method aims to effectively identify and utilize distinguishing features for deepfake detection.
  arXiv  Detail & Related papers  (2022-09-12T17:17:49Z)
• Weakly-Supervised Action Detection Guided by Audio Narration [50.4318060593995]
  We propose a model to learn from the narration supervision and utilize multimodal features, including RGB, motion flow, and ambient sound. Our experiments show that noisy audio narration suffices to learn a good action detection model, thus reducing annotation expenses.
  arXiv  Detail & Related papers  (2022-05-12T06:33:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.