Metadata-Aware Multi-Prompt Reasoning for Zero-Shot Accident Understanding

• URL: http://arxiv.org/abs/2606.12047v1
• Date: Wed, 10 Jun 2026 13:12:40 GMT
• Title: Metadata-Aware Multi-Prompt Reasoning for Zero-Shot Accident Understanding
• Abstract summary: We propose a three-stage pipeline that decomposes the accident understanding into when, what, and where.<n>We show that decomposing zero-shot video understanding into temporal localization, semantic classification, and spatial grounding enable more reliable reasoning.
• Score: 9.094828055298166
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we address the problem of zero-shot understanding of accidents from surveillance videos by identifying when an impact event occurs, what type of impact it is, and where in the frame it occurs using natural language. We propose a three-stage pipeline that decomposes the accident understanding into when, what, and where. The first stage extracts a short temporal window around the impact using vision-language similarity. In the second stage, we perform metadata-driven multi-prompt reasoning with five complementary views (baseline, motion, geometry, contrast, and tiebreaker) and resolve disagreement via an entropy-gated pairwise adjudicator. Finally, we localize the impact of an open-vocabulary detector queried on the predicted accident type and scene layout, and aggregate detections across keyframes using a score-weighted centroid. Our pipeline achieves a substantial improvement in the harmonic-mean score over a centre-of-frame baseline on the zero-shot ACCIDENT @ CVPR benchmark. We show that decomposing zero-shot video understanding into temporal localization, semantic classification, and spatial grounding enable more reliable reasoning with vision-language models than direct prompting alone.

Related papers

• SynCrash: A Multi-Stage Pipeline for Zero-Shot Accident Detection and Localization in Traffic Surveillance Video [0.0]
  We present SynCrash, a pipeline for zero-shot accident detection, spatial localization, and collision-type classification in CCTV video.<n>Our approach addresses the ACCIDENT at CVPR 2026 Challenge, which requires predicting when an accident occurs, where in the frame the impact happens, and what type of collision it is.
  arXiv  Detail & Related papers  (2026-08-30T12:42:59Z)
• Evidence of Absence: Cross-Modal Abductive Risk Perception to Sustain World Models When Vision Fails [11.206758778146282]
  A structured world-state is designed to preserve prediction-critical content when perception degrades, but it presumes observations to populate it.<n>We address how to sustain the world model from a complementary modality by treating the absence of expected co-evidence as evidence of a hidden cause.<n>A microphone-array front-end estimates the bearing of engine and tire sources and extracts approach-rate evidence.
  arXiv  Detail & Related papers  (2026-08-15T00:38:17Z)
• Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Models [154.39583176906893]
  Large language models (MLs) have advanced understanding, they remain highly prone to hallucinations in scenes.<n>We argue this stems from a failure intemporal monitoring, the ability to persistently track object dynamic identities, states, and relations over time.<n>Existing benchmarks obscure this deficit relying on single final-answer queries that can often be resolved via local visual cues or statistical priors.
  arXiv  Detail & Related papers  (2026-05-09T14:32:36Z)
• A Unified Reasoning Framework for Holistic Zero-Shot Video Anomaly Analysis [64.42659342276117]
  Most video-anomaly research stops at frame-wise detection, offering little insight into why an event is abnormal.<n>Recent video anomaly localization and video anomaly understanding methods improve explainability but remain data-dependent and task-specific.<n>We propose a unified reasoning framework that bridges the gap between temporal detection, spatial localization, and textual explanation.
  arXiv  Detail & Related papers  (2025-11-02T14:49:08Z)
• TRACES: Temporal Recall with Contextual Embeddings for Real-Time Video Anomaly Detection [0.0]
  This paper addresses the context-aware zero-shot anomaly detection challenge.<n>Our approach defines a memory-augmented pipeline, correlating temporal signals with visual embeddings.<n>We achieve 90.4% AUC on UCF-Crime and 83.67% AP on XD-Violence, a new state-of-the-art among zero-shot models.
  arXiv  Detail & Related papers  (2025-11-01T14:54:08Z)
• Representation-Level Counterfactual Calibration for Debiased Zero-Shot Recognition [36.36218470387896]
  We recast this issue as a causal inference problem and ask: Would the prediction remain if the object appeared in a different environment?<n>To answer this at inference time, we estimate object and background expectations within CLIP's representation space, and synthesize counterfactual embeddings.<n>Without retraining or prompt design, our method substantially improves both worst-group and average accuracy on context-sensitive benchmarks.
  arXiv  Detail & Related papers  (2025-10-30T13:11:23Z)
• Causality Matters: How Temporal Information Emerges in Video Language Models [17.570777893613137]
  We find that removing or modifying positional encodings in video inputs yields minimal degradation in the performance of temporal understanding.<n>To explain this behavior, we conduct substantial analysis experiments to trace how temporal information is integrated within the model.<n>We propose two efficiency-oriented strategies: staged cross-modal attention and a temporal exit mechanism for early token truncation.
  arXiv  Detail & Related papers  (2025-08-15T16:33:14Z)
• Autoregressive Denoising Score Matching is a Good Video Anomaly Detector [36.96911195723131]
  Video anomaly detection (VAD) is an important computer vision problem.<n>We introduce a noise-conditioned score transformer for denoising score matching.<n>Then, we introduce a scene-dependent and motion-aware score function.<n>We integrate unaffected visual information via a novel autoregressive denoising score matching mechanism.
  arXiv  Detail & Related papers  (2025-06-29T15:14:32Z)
• Context-aware TFL: A Universal Context-aware Contrastive Learning Framework for Temporal Forgery Localization [60.73623588349311]
  We propose a universal context-aware contrastive learning framework (UniCaCLF) for temporal forgery localization.<n>Our approach leverages supervised contrastive learning to discover and identify forged instants by means of anomaly detection.<n>An efficient context-aware contrastive coding is introduced to further push the limit of instant feature distinguishability between genuine and forged instants.
  arXiv  Detail & Related papers  (2025-06-10T06:40:43Z)
• Spatial-Frequency Discriminability for Revealing Adversarial Perturbations [53.279716307171604]
  Vulnerability of deep neural networks to adversarial perturbations has been widely perceived in the computer vision community. Current algorithms typically detect adversarial patterns through discriminative decomposition for natural and adversarial data. We propose a discriminative detector relying on a spatial-frequency Krawtchouk decomposition.
  arXiv  Detail & Related papers  (2023-05-18T10:18:59Z)
• Weakly-Supervised Video Object Grounding via Causal Intervention [82.68192973503119]
  We target at the task of weakly-supervised video object grounding (WSVOG), where only video-sentence annotations are available during model learning. It aims to localize objects described in the sentence to visual regions in the video, which is a fundamental capability needed in pattern analysis and machine learning.
  arXiv  Detail & Related papers  (2021-12-01T13:13:03Z)
• Deconfounded Video Moment Retrieval with Causal Intervention [80.90604360072831]
  We tackle the task of video moment retrieval (VMR), which aims to localize a specific moment in a video according to a textual query. Existing methods primarily model the matching relationship between query and moment by complex cross-modal interactions. We propose a causality-inspired VMR framework that builds structural causal model to capture the true effect of query and video content on the prediction.
  arXiv  Detail & Related papers  (2021-06-03T01:33:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.