PrismVAU: Prompt-Refined Inference System for Multimodal Video Anomaly Understanding

• URL: http://arxiv.org/abs/2601.02927v2
• Date: Wed, 07 Jan 2026 08:16:35 GMT
• Title: PrismVAU: Prompt-Refined Inference System for Multimodal Video Anomaly Understanding
• Abstract summary: Video Anomaly Understanding (VAU) extends traditional Video Anomaly Detection (VAD) by describing and reasoning about their context.<n>Existing VAU approaches often rely on fine-tuned multimodal large language models (MLLMs) or external modules such as video captioners.<n>We introduce PrismVAU, a lightweight yet effective system for real-time VAU that leverages a single off-the-shelf MLLM for anomaly scoring, explanation, and prompt optimization.
• Score: 48.0087455344611
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Video Anomaly Understanding (VAU) extends traditional Video Anomaly Detection (VAD) by not only localizing anomalies but also describing and reasoning about their context. Existing VAU approaches often rely on fine-tuned multimodal large language models (MLLMs) or external modules such as video captioners, which introduce costly annotations, complex training pipelines, and high inference overhead. In this work, we introduce PrismVAU, a lightweight yet effective system for real-time VAU that leverages a single off-the-shelf MLLM for anomaly scoring, explanation, and prompt optimization. PrismVAU operates in two complementary stages: (1) a coarse anomaly scoring module that computes frame-level anomaly scores via similarity to textual anchors, and (2) an MLLM-based refinement module that contextualizes anomalies through system and user prompts. Both textual anchors and prompts are optimized with a weakly supervised Automatic Prompt Engineering (APE) framework. Extensive experiments on standard VAD benchmarks demonstrate that PrismVAU delivers competitive detection performance and interpretable anomaly explanations -- without relying on instruction tuning, frame-level annotations, and external modules or dense processing -- making it an efficient and practical solution for real-world applications.

Related papers

• ESOM: Efficiently Understanding Streaming Video Anomalies with Open-world Dynamic Definitions [27.912128185225054]
  Open-world video anomaly detection (OWVAD) aims to detect and explain abnormal events under different anomaly definitions.<n>Recent MLLM-based methods have shown promising open-world generalization, but still suffer from three major limitations.<n>This paper proposes ESOM, an efficient streaming OWVAD model that operates in a training-free manner.
  arXiv  Detail & Related papers  (2026-04-09T03:51:14Z)
• QVAD: A Question-Centric Agentic Framework for Efficient and Training-Free Video Anomaly Detection [11.652235163237117]
  Video Anomaly Detection (VAD) is a fundamental challenge in computer vision.<n>We propose QVAD, a question-centric agentic framework that treats VLM-LLM interaction as a dynamic dialogue.<n>QVAD achieves high inference speeds with minimal memory footprints, making advanced VAD capabilities deployable on resource-constrained edge devices.
  arXiv  Detail & Related papers  (2026-04-03T13:48:34Z)
• Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs [61.64185573373394]
  We propose a training-free framework that uses an MLLM's intrinsic uncertainty as a proactive guidance signal.<n>We introduce a unified mechanism that scores candidate visual inputs by response uncertainty, enabling the model to autonomously focus on the most salient data.<n>Our work validates that harnessing intrinsic uncertainty is a powerful, general strategy for enhancing fine-grained multimodal performance.
  arXiv  Detail & Related papers  (2025-10-01T09:20:51Z)
• HiProbe-VAD: Video Anomaly Detection via Hidden States Probing in Tuning-Free Multimodal LLMs [8.18063726177317]
  Video Anomaly Detection (VAD) aims to identify and locate deviations from normal patterns in video sequences.<n>We propose HiProbe-VAD, a novel framework that leverages pre-trained Multimodal Large Language Models (MLLMs) for VAD without requiring fine-tuning.
  arXiv  Detail & Related papers  (2025-07-23T10:41:46Z)
• ONLY: One-Layer Intervention Sufficiently Mitigates Hallucinations in Large Vision-Language Models [67.75439511654078]
  Large Vision-Language Models (LVLMs) have introduced a new paradigm for understanding and reasoning about image input through textual responses.<n>They face the persistent challenge of hallucination, which introduces practical weaknesses and raises concerns about their reliable deployment in real-world applications.<n>We propose ONLY, a training-free decoding approach that requires only a single query and a one-layer intervention during decoding, enabling efficient real-time deployment.
  arXiv  Detail & Related papers  (2025-07-01T16:01:08Z)
• Reasoning-Aligned Perception Decoupling for Scalable Multi-modal Reasoning [95.44766931218896]
  Multi-modal large language models (MLLMs) still lag behind text-based reasoning.<n>We introduce Perception-Reasoning Decoupling, which modularizes the MLLM's reasoning component and makes it easily replaceable.<n>We propose a novel reinforcement learning algorithm called Visual Perception Optimization (VPO) to align the MLLM's perceptual output with the final reasoning task.
  arXiv  Detail & Related papers  (2025-06-05T02:28:07Z)
• MemoryOut: Learning Principal Features via Multimodal Sparse Filtering Network for Semi-supervised Video Anomaly Detection [30.470777079947958]
  Video Anomaly Detection (VAD) methods based on reconstruction or prediction face two critical challenges.<n>Strong generalization capability often results in accurate reconstruction or prediction of abnormal events.<n>reliance only on low-level appearance and motion cues limits their ability to identify high-level semantic in abnormal events from complex scenes.
  arXiv  Detail & Related papers  (2025-06-03T07:14:57Z)
• Vad-R1: Towards Video Anomaly Reasoning via Perception-to-Cognition Chain-of-Thought [58.321044666612174]
  Vad-R1 is an end-to-end MLLM-based framework for Video Anomaly Reasoning.<n>We design a Perception-to-Cognition Chain-of-Thought (P2C-CoT) that simulates the human process of recognizing anomalies.<n>We also propose an improved reinforcement learning algorithm AVA-GRPO, which explicitly incentivizes the anomaly reasoning capability of MLLMs.
  arXiv  Detail & Related papers  (2025-05-26T12:05:16Z)
• Prism: A Framework for Decoupling and Assessing the Capabilities of VLMs [83.24033574914425]
  We present Prism, an innovative framework designed to disentangle the perception and reasoning processes involved in visual question solving. Prism comprises two distinct stages: a perception stage that utilizes a VLM to extract and articulate visual information in textual form, and a reasoning stage that formulates responses based on the extracted visual information. Our analytical framework provides several valuable insights, underscoring Prism's potential as a cost-effective solution for vision-language tasks.
  arXiv  Detail & Related papers  (2024-06-20T17:54:03Z)
• Holmes-VAD: Towards Unbiased and Explainable Video Anomaly Detection via Multi-modal LLM [35.06386971859359]
  Holmes-VAD is a novel framework that leverages precise temporal supervision and rich multimodal instructions. We construct the first large-scale multimodal VAD instruction-tuning benchmark, VAD-Instruct50k. Building upon the VAD-Instruct50k dataset, we develop a customized solution for interpretable video anomaly detection.
  arXiv  Detail & Related papers  (2024-06-18T03:19:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.