VCBench: A Streaming Counting Benchmark for Spatial-Temporal State Maintenance in Long Videos

• URL: http://arxiv.org/abs/2603.12703v1
• Date: Fri, 13 Mar 2026 06:28:46 GMT
• Title: VCBench: A Streaming Counting Benchmark for Spatial-Temporal State Maintenance in Long Videos
• Abstract summary: We propose VCBench, a streaming counting benchmark that repositions counting as a minimal probe for diagnosing world state maintenance capability.<n> VCBench contains 406 videos with frame-by-frame annotations of 10,071 event occurrence moments and object state change moments.<n>We design three complementary metrics to diagnose numerical precision, trajectory consistency, and temporal awareness.
• Score: 20.761149440325614
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video understanding requires models to continuously track and update world state during playback. While existing benchmarks have advanced video understanding evaluation across multiple dimensions, the observation of how models maintain world state remains insufficient. We propose VCBench, a streaming counting benchmark that repositions counting as a minimal probe for diagnosing world state maintenance capability. We decompose this capability into object counting (tracking currently visible objects vs.\ tracking cumulative unique identities) and event counting (detecting instantaneous actions vs.\ tracking complete activity cycles), forming 8 fine-grained subcategories. VCBench contains 406 videos with frame-by-frame annotations of 10,071 event occurrence moments and object state change moments, generating 1,000 streaming QA pairs with 4,576 query points along timelines. By observing state maintenance trajectories through streaming multi-point queries, we design three complementary metrics to diagnose numerical precision, trajectory consistency, and temporal awareness. Evaluation on mainstream video-language models shows that current models still exhibit significant deficiencies in spatial-temporal state maintenance, particularly struggling with tasks like periodic event counting. VCBench provides a diagnostic framework for measuring and improving state maintenance in video understanding systems.

Related papers

• StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding [42.88155526332792]
  We introduce StreamArena, a benchmark for hour-scale, interactive streaming video understanding.<n> evaluation across diverse systems exposes a tension between continuous interaction and long-horizon multimodal comprehension.<n>We address this tension with StreamMind, a two-tier architecture that assigns latency-critical interaction and proactive monitoring to independently scheduled workers.
  arXiv  Detail & Related papers  (2026-08-06T07:46:37Z)
• Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events [52.031070006859544]
  Video multimodal large language models (MLLMs) have made rapid progress on general and long-form video understanding, yet their ability to preserve brief answer-critical visual evidence remains underexplored.<n>We introduce Moment-Video, a benchmark for diagnosing the temporal fidelity of video MLLMs through momentary visual event understanding.
  arXiv  Detail & Related papers  (2026-06-01T17:32:20Z)
• TOC-Bench: A Temporal Object Consistency Benchmark for Video Large Language Models [9.648992690108086]
  Video large language models (Video-LLMs) have made strong progress in general video understanding, but their ability to maintain temporal object consistency remains underexplored.<n>We introduce TOC-Bench, a diagnostic benchmark for evaluating temporal object consistency in Video-LLMs.
  arXiv  Detail & Related papers  (2026-05-11T02:47:59Z)
• VSAS-BENCH: Real-Time Evaluation of Visual Streaming Assistant Models [39.78979236902648]
  Streaming vision-language models (VLMs) continuously generate responses given an instruction prompt and an online stream of input frames.<n>We propose VSAS-Bench, a new framework and benchmark for Visual Streaming Assistants.
  arXiv  Detail & Related papers  (2026-04-08T22:31:20Z)
• VC-Bench: Pioneering the Video Connecting Benchmark with a Dataset and Evaluation Metrics [83.61875204972465]
  We introduce Video Connecting, a task that aims to generate smooth intermediate video content between given start and end clips.<n>To bridge this gap, we proposed VC-Bench, a novel benchmark specifically designed for video connecting.<n> VC-Bench focuses on three core aspects: Video Quality Score VQS, Start-End Consistency Score SECS, and Transition Smoothness Score TSS.
  arXiv  Detail & Related papers  (2026-01-27T06:15:12Z)
• Convolutions Need Registers Too: HVS-Inspired Dynamic Attention for Video Quality Assessment [1.7188280334580195]
  We present Dynamic Attention with Global Registers for Video Quality Assessment (DAGR-VQA), the first framework integrating register-token directly into a convolutional backbone for dynamic video saliency prediction.<n>Our model integrates dynamic saliency maps with RGB inputs, capturing spatial data and analyzing it through a temporal transformer to deliver a perceptually consistent video quality assessment.
  arXiv  Detail & Related papers  (2026-01-16T07:25:16Z)
• SVAG-Bench: A Large-Scale Benchmark for Multi-Instance Spatio-temporal Video Action Grounding [48.64661382961745]
  We introduce Spatio-temporal Video Action Grounding (SVAG), a novel task that requires models to simultaneously detect, track, and temporally localize all referent objects in videos.<n>To support this task, we construct SVAG-Bench, a large-scale benchmark comprising 688 videos, 19,590 annotated records, and 903 unique verbs.<n> Empirical results show that existing models perform poorly on SVAG, particularly in dense or complex scenes.
  arXiv  Detail & Related papers  (2025-10-14T22:10:49Z)
• SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understanding [56.78088668917983]
  We introduce SVBench, a pioneering benchmark with temporal multi-turn question-answering chains.<n>We design a semi-automated annotation pipeline to obtain 49,979 Question-Answer (QA) pairs of 1,353 streaming videos.<n>Our experimental results, obtained from 14 models in dialogue and streaming evaluations, reveal that while the closed-source GPT-4o outperforms others, most open-source LVLMs struggle with long-context streaming video understanding.
  arXiv  Detail & Related papers  (2025-02-15T14:29:44Z)
• Track-On: Transformer-based Online Point Tracking with Memory [34.744546679670734]
  We introduce Track-On, a simple transformer-based model designed for online long-term point tracking.<n>Unlike prior methods that depend on full temporal modeling, our model processes video frames causally without access to future frames.<n>At inference time, it employs patch classification and refinement to identify correspondences and track points with high accuracy.
  arXiv  Detail & Related papers  (2025-01-30T17:04:11Z)
• TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Models [75.42002690128486]
  TemporalBench is a new benchmark dedicated to evaluating fine-grained temporal understanding in videos. It consists of 10K video question-answer pairs, derived from 2K high-quality human annotations detailing the temporal dynamics in video clips. Results show that state-of-the-art models like GPT-4o achieve only 38.5% question answering accuracy on TemporalBench.
  arXiv  Detail & Related papers  (2024-10-14T17:59:58Z)
• Argus++: Robust Real-time Activity Detection for Unconstrained Video Streams with Overlapping Cube Proposals [85.76513755331318]
  Argus++ is a robust real-time activity detection system for analyzing unconstrained video streams. The overall system is optimized for real-time processing on standalone consumer-level hardware.
  arXiv  Detail & Related papers  (2022-01-14T03:35:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.