Parser-Free VLM Verification for Federated Weakly Supervised Video Anomaly Detection

• URL: http://arxiv.org/abs/2609.07455v1
• Date: Mon, 07 Sep 2026 13:04:00 GMT
• Title: Parser-Free VLM Verification for Federated Weakly Supervised Video Anomaly Detection
• Abstract summary: We introduce a lightweight federated MIL-VLM cascade in which only a compact MIL scorer is trained across clients, while a frozen VLM verifies high-scoring suspect segments post hoc.<n>We study two VLM feedback interfaces: parsed text-generation decisions and a logit-based interface that extracts a continuous anomaly score from next-token Yes/No probabilities.<n>In contrast, the logit interface provides a fixed-free signal that improves both frame-level AUC and frame-level AP over the MIL baseline across both VLMs.
• Score: 0.27998963147546135
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: How can vision-language models help video anomaly detection (VAD) when surveillance data remain distributed, weakly labeled, and resource-constrained? Most weakly supervised VAD methods assume centralized training; recent VLM-based extensions further rely on dense inference, generated explanations, or additional adaptation. We introduce a lightweight federated MIL-VLM cascade in which only a compact MIL scorer is trained across clients, while a frozen VLM verifies high-scoring suspect segments post hoc. We study two VLM feedback interfaces: parsed text-generation decisions and a logit-based interface that extracts a continuous anomaly score from next-token Yes/No probabilities. Experiments on UCF-Crime with InternVL3.5-2B and Qwen3-VL-2B-Instruct show that text-generation verification can improve frame-level AUC after diagnostic temporal post-processing, but remains sensitive to prompts, parsers, model choice, and smoothing. In contrast, the logit interface provides a fixed parser-free signal that improves both frame-level AUC and frame-level AP over the MIL baseline across both VLMs, without temporal post-processing in its main configuration. Since suspect segments are updated independently once available, next-token logit feedback provides a simple segment-local alternative to text-generation verification.

Related papers

• Probe-VAD: Ordinal Likelihood Probing for Training-Free Video Anomaly Detection [34.268676888953046]
  Video anomaly detection (VAD) aims to localize anomalous events in untrimmed videos.<n>Existing approaches impose restrictive interfaces between visual understanding and anomaly scoring.<n>We propose textbfProbe-VAD, an ordinal binary-probing framework that directly probes severity preferences from a frozen VLM.
  arXiv  Detail & Related papers  (2026-09-15T13:59:47Z)
• COPRA: Conditional Parameter Adaptation with Reinforcement Learning for Video Anomaly Detection [10.332480827185973]
  Vision-language models (VLMs) have shown strong performance in video anomaly detection (VAD)<n>Existing VLM-based VAD methods suffer from a fundamental mismatch between training and inference.<n>We propose COPRA, a conditional parameter adaptation framework for VLM-based VAD.
  arXiv  Detail & Related papers  (2026-05-14T18:39:40Z)
• Fast-dVLM: Efficient Block-Diffusion VLM via Direct Conversion from Autoregressive VLM [58.322826487307765]
  We present Fast-dVLM, a block-diffusion-based VLM that enables KV-cache-compatible parallel decoding and speculative block decoding for inference acceleration.<n>With SGLang integration and FP8 quantization, Fast-dVLM achieves over 6x end-to-end inference speedup over the AR baseline.
  arXiv  Detail & Related papers  (2026-04-08T08:50:08Z)
• RVLM: Recursive Vision-Language Models with Adaptive Depth [0.1631115063641726]
  RVLM replaces single-pass inference with an iterative generate-execute loop.<n>Every diagnostic claim is grounded in executable code, satisfying auditability requirements.<n>We evaluate on BraTS 2023 Meningioma (brain MRI) and MIMIC-CXR (chest X-ray) using Gemini 2.5 Flash without fine-tuning.
  arXiv  Detail & Related papers  (2026-03-25T11:52:55Z)
• DV-VLN: Dual Verification for Reliable LLM-Based Vision-and-Language Navigation [18.493700097379186]
  Vision-and-Language Navigation (VLN) requires an embodied agent to navigate in a complex 3D environment according to natural language instructions.<n>Recent progress in large language models (LLMs) has enabled language-driven navigation with improved interpretability.<n>We propose DV-VLN, a new VLN framework that follows a generate-then-verify paradigm.
  arXiv  Detail & Related papers  (2026-01-26T13:47:55Z)
• VideoHEDGE: Entropy-Based Hallucination Detection for Video-VLMs via Semantic Clustering and Spatiotemporal Perturbations [4.509454543418357]
  Hallucinations in video-capable vision models (VideoVLMs) remain frequent and high-confidence.<n>We introduce VideoHEDGE, a modular framework for hallucination detection in question answering.
  arXiv  Detail & Related papers  (2026-01-13T13:42:05Z)
• 1 + 1 > 2: Detector-Empowered Video Large Language Model for Spatio-Temporal Grounding and Reasoning [53.28271278708241]
  We present a Detector-Empowered Video LLM, short for DEViL.<n> DEViL couples a Video LLM with an open-vocabulary detector (OVD)<n>Unlike tokens that merely serve as spatial prompts or segmentor switches, the RST functions as both a control signal and a replacement for the OVD's text embedding.
  arXiv  Detail & Related papers  (2025-12-07T06:11:15Z)
• Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policies [62.653984010274485]
  Vision-Language-Action (VLA) models adapt large vision-language backbones to map images and instructions into robot actions.<n> prevailingAs either generate actions auto-regressively in a fixed left-to-right order or attach separate or diffusion heads outside the backbone.<n>We present Discrete Diffusion VLA, a unified-transformer policy that models discretized action chunks with discrete diffusion.
  arXiv  Detail & Related papers  (2025-08-27T17:39:11Z)
• Video-Level Language-Driven Video-Based Visible-Infrared Person Re-Identification [47.40091830500585]
  Video-based Visible-basedInfrared Person Re-Identification (VVIReID) aims to match pedestrian sequences across modalities by extracting modality-in sequence-level features.<n>A framework, video-level language-driven VVI-ReID (VLD), consists of two core modules: inmodality language (IMLP) and spatialtemporal aggregation.
  arXiv  Detail & Related papers  (2025-06-03T04:49:08Z)
• OVLW-DETR: Open-Vocabulary Light-Weighted Detection Transformer [63.141027246418]
  We propose Open-Vocabulary Light-Weighted Detection Transformer (OVLW-DETR), a deployment friendly open-vocabulary detector with strong performance and low latency. We provide an end-to-end training recipe that transferring knowledge from vision-language model (VLM) to object detector with simple alignment. Experimental results demonstrate that the proposed approach is superior over existing real-time open-vocabulary detectors on standard Zero-Shot LVIS benchmark.
  arXiv  Detail & Related papers  (2024-07-15T12:15:27Z)
• Can Large Vision-Language Models Correct Semantic Grounding Errors By Themselves? [61.899791071654654]
  We investigate whether Vision-Language Models (VLMs) can improve their semantic grounding by "receiving" feedback.<n>We find that if prompted appropriately, VLMs can utilize feedback both in a single step and iteratively.<n>We show grounding accuracy consistently improves using automated feedback across all models in all settings investigated.
  arXiv  Detail & Related papers  (2024-04-09T17:59:04Z)
• F-VLM: Open-Vocabulary Object Detection upon Frozen Vision and Language Models [54.21757555804668]
  We present F-VLM, a simple open-vocabulary object detection method built upon Frozen Vision and Language Models. F-VLM simplifies the current multi-stage training pipeline by eliminating the need for knowledge distillation or detection-tailored pretraining.
  arXiv  Detail & Related papers  (2022-09-30T17:59:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.