VideoZeroBench: Probing the Limits of Video MLLMs with Spatio-Temporal Evidence Verification

• URL: http://arxiv.org/abs/2604.01569v1
• Date: Thu, 02 Apr 2026 03:29:43 GMT
• Title: VideoZeroBench: Probing the Limits of Video MLLMs with Spatio-Temporal Evidence Verification
• Abstract summary: VideoBench is a hierarchical benchmark for long-video answering that rigorously verifies evidence.<n>It comprises 500 manually annotated questions across 13 domains, paired with temporal intervals and spatial bounding boxes as evidence.<n>Experiments show that even GeminiPro-3-Pro correctly answers fewer than 17% of questions under the standard end-to-end QA setting.<n>Results expose a significant gap between surface-level answer correctness and genuine evidence-based reasoning.
• Score: 73.02304272829785
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent video multimodal large language models achieve impressive results across various benchmarks. However, current evaluations suffer from two critical limitations: (1) inflated scores can mask deficiencies in fine-grained visual understanding and reasoning, and (2) answer correctness is often measured without verifying whether models identify the precise spatio-temporal evidence supporting their predictions. To address this, we present VideoZeroBench, a hierarchical benchmark designed for challenging long-video question answering that rigorously verifies spatio-temporal evidence. It comprises 500 manually annotated questions across 13 domains, paired with temporal intervals and spatial bounding boxes as evidence. To disentangle answering generation, temporal grounding, and spatial grounding, we introduce a five-level evaluation protocol that progressively tightens evidence requirements. Experiments show that even Gemini-3-Pro correctly answers fewer than 17% of questions under the standard end-to-end QA setting (Level-3). When grounding constraints are imposed, performance drops sharply: No model exceeds 1% accuracy when both correct answering and accurate spatio-temporal localization are required (Level-5), with most failing to achieve any correct grounded predictions. These results expose a significant gap between surface-level answer correctness and genuine evidence-based reasoning, revealing that grounded video understanding remains a bottleneck for long-video QA. We further analyze performance across minimal evidence spans, atomic abilities, and inference paradigms, providing insights for future research in grounded video reasoning. The benchmark and code will be made publicly available.

Related papers

• Video-HolmesV2: Can MLLMs Reason with Spatio-Temporal Audio-Visual Evidence in Long Videos? [22.544360578469156]
  We present Video-Holmes-V2, a novel benchmark for Deep Audio-Visual Coupling.<n>It enforces an Evidence-Based Evaluation, requiring models to justify answers with precise-temporal audio-visual evidence.<n>Our method distills high-value reasoning cues to mitigate long-grained noise.
  arXiv  Detail & Related papers  (2026-09-15T14:30:00Z)
• Evidence-Backed Video Question Answering [90.93225758130426]
  Existing explainability efforts rely on textual rationales or sparse bounding boxes.<n>We propose Evidence-Backed Video Question Answering (E-VQA)<n>To support this, we introduce ST-Evidence, the first human-verified benchmark for both discriminative and generative pixel-level grounding.
  arXiv  Detail & Related papers  (2026-07-13T17:49:10Z)
• EG-VQA: Benchmarking Verifiable Video Question Answering with Grounded Temporal Evidence [48.314554375516366]
  This paper introduces the Evidence-Grounded Video Question Answering Benchmark (EG-VQA)<n> EG-VQA is comprised of 2,067 videos and 11,838 QA pairs with fine-grained evidence annotations.<n>We show that even strong proprietary models struggle to accurately ground their predictions.<n>We propose EG-Reasoner, an evidence-grounded reasoning model trained with explicit supervision.
  arXiv  Detail & Related papers  (2026-06-23T16:49:28Z)
• SER: Learning to Ground Video Reasoning with Semantic Evidence Rewards [8.705689021755221]
  We propose Semantic Evidence Reward (SER) to reform RL as a constrained task.<n>Instead of computing pixel-level annotations, SER uses a referee enhancing a VLM to evaluate model-generated evidence claims.<n>On the VLM benchmark, SER achieves 49.6%, improving by 3.0 points over the strong evidence-ground baseline.
  arXiv  Detail & Related papers  (2026-06-23T15:50:33Z)
• Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events [52.031070006859544]
  Video multimodal large language models (MLLMs) have made rapid progress on general and long-form video understanding, yet their ability to preserve brief answer-critical visual evidence remains underexplored.<n>We introduce Moment-Video, a benchmark for diagnosing the temporal fidelity of video MLLMs through momentary visual event understanding.
  arXiv  Detail & Related papers  (2026-06-01T17:32:20Z)
• HERBench: A Benchmark for Multi-Evidence Integration in Video Question Answering [13.370338205427911]
  We present HERBench, a VideoQA benchmark to assess multi-evidence integration across time.<n>HERBench comprises 26K five-way multiple-choice questions organized into twelve compositional tasks.<n>We show that HERBench imposes substantially higher demand than prior datasets.
  arXiv  Detail & Related papers  (2025-12-16T19:34:47Z)
• Know-Show: Benchmarking Video-Language Models on Spatio-Temporal Grounded Reasoning [18.15310805625469]
  We present Know-Show, a new benchmark designed to evaluate multimodal Video-Language Models (Video-LMs)<n>Know-Show unifies reasoning and localization within a single evaluation framework consisting of five scenarios across spatial (person, object, person-object, and hand-object) and temporal dimensions.<n>Built from Charades, Action Genome, and Ego4D with 2.5K human-language questions, the benchmark exposes significant gaps between current Video-LMs and human reasoning.<n>To bridge this gap, we propose GRAM, a training-free plug-in that augments Video-LMs with fine-grained grounding
  arXiv  Detail & Related papers  (2025-12-05T08:15:49Z)
• Open-o3 Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidence [70.2803680525165]
  We introduce Open-o3 Video, a non-agent framework that integrates explicit evidence into video reasoning.<n>The model highlights key objects and bounding boxes alongside its answers, allowing reasoning to be grounded in concrete visual observations.<n>On V-STAR benchmark, Open-o3 Video achieves state-of-the-art performance, raising mAM by 14.4% and mL timestamp by 24.2%.
  arXiv  Detail & Related papers  (2025-10-23T14:05:56Z)
• Video SimpleQA: Towards Factuality Evaluation in Large Video Language Models [77.96693360763925]
  Video SimpleQA is the first comprehensive benchmark tailored for factuality evaluation in video contexts.<n>Our work differs from existing video benchmarks through the following key features: Knowledge required: demanding integration of external knowledge beyond the video's explicit narrative.<n>Short-form definitive answer: Answers are crafted as unambiguous and definitively correct in a short format with minimal scoring variance.
  arXiv  Detail & Related papers  (2025-03-24T17:46:09Z)
• TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Models [75.42002690128486]
  TemporalBench is a new benchmark dedicated to evaluating fine-grained temporal understanding in videos. It consists of 10K video question-answer pairs, derived from 2K high-quality human annotations detailing the temporal dynamics in video clips. Results show that state-of-the-art models like GPT-4o achieve only 38.5% question answering accuracy on TemporalBench.
  arXiv  Detail & Related papers  (2024-10-14T17:59:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.