BVB: Benchmarking Agentic Video Understanding via Programmatic Reconstruction in Blender
- URL: http://arxiv.org/abs/2609.15478v1
- Date: Mon, 14 Sep 2026 12:36:05 GMT
- Title: BVB: Benchmarking Agentic Video Understanding via Programmatic Reconstruction in Blender
- Abstract summary: We test the ability of agents to reconstruct video by asking them to reconstruct realworld videos as animated Blender scenes.<n>The benchmark renders each reconstruction from its animated camera and evaluates it on two axes: (1) Dual VQA measures how many facts the reconstruction preserves, and (2) Latent Similarity measures how closely reconstruction matches source video perceptually.<n>The best model 88.6 Latent Similarity reaches but retains only 53.7% of the source-correcttemporal answers.
- Score: 46.08439123011388
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Multimodal agents can create complex videos in software such as Blender by coding without relying on diffusion models. Yet video understanding benchmarks still evaluate models mainly through question answering. If an agent truly understands a video, it can reconstruct it programmatically. We introduce BVB, Blender-VideoBench, a benchmark that tests this ability by asking agents to reconstruct real-world videos as animated Blender scenes. To ensure fair comparison, each agent programs the reconstruction through a lightweight harness, Mini-BVB, in an identical sandbox under a shared cost limit. The benchmark renders each reconstruction from its animated camera and evaluates it on two axes: (1) Dual VQA measures how many spatiotemporal facts the reconstruction preserves. (2) Latent Similarity measures how closely the reconstruction matches the source video perceptually. Our overall score, a square-root mean, favors balanced performance. We evaluate 51 configurations from 10 model families and analyze semantic retention, perceptual similarity, reasoning effort, and cost. The best model reaches 88.6 Latent Similarity but retains only 53.7% of the source-correct spatiotemporal answers. Additional reasoning improves visual similarity but does not close this gap in factual accuracy. In a blind study with 15 raters and five configurations, Latent Similarity correlates strongly with human preference. These results show that programmatic reconstruction is a viable test of agentic video understanding, and that semantic retention remains the main challenge.
Related papers
- VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine System [77.32035315337522]
Text-video models struggle to generate physically consistent dynamics the temporal evolution of a scene must be inferred implicitly from a compressed text prompt.<n>Existing chain-of-thought approaches introduce intermediate plans or visual states, but these representations are typically non-cutable or temporally sparse.<n>We introduce VideoCo, an agentic dual-engine framework in which executable Blender code serves as a process-level chain of thought.
arXiv Detail & Related papers (2026-07-29T18:38:23Z) - ReConFuse: Reconstruction-Error Guided Semantic Fusion for AI-Generated Video Detection [10.547980601243518]
We propose ReConFuse, a reconstruction-guided semantic fusion framework for video-level AI-generated video detection.<n>ReConFuse extracts reconstruction error cues from WF-VAE reconstructed videos, aligns them with multi-frame semantic features, and uses a Mamba-based module to model temporal evolution for video-level classification.
arXiv Detail & Related papers (2026-06-03T10:35:47Z) - WorldJen: An End-to-End Multi-Dimensional Benchmark for Generative Video Models [0.8640786765448132]
WorldJen is a framework for evaluating generative video models.<n>A blind human preference study is conducted to accumulate 2,696 pairwise annotations from 7 annotators.<n>A VLM-as-a-judge evaluation engine using prompt-specific, dimension-specific Likert questions is developed.
arXiv Detail & Related papers (2026-05-05T08:03:34Z) - Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding [98.3098451637867]
Video-MME-v2 is a comprehensive benchmark designed to rigorously evaluate the robustness and faithfulness of video understanding.<n>To guarantee data quality, Video-MME-v2 is constructed through a rigorously controlled human annotation pipeline.
arXiv Detail & Related papers (2026-04-06T17:59:56Z) - Video-Based Reward Modeling for Computer-Use Agents [40.27314571412647]
We study reward modeling from execution video: a sequence of sequences from an agent trajectory independent of the agent's internal reasoning or actions.<n>We introduce Execution Video Reward 53k (ExeVR-53k), a dataset of 53k high-quality video--task--reward triplets.<n>Building on these components, we fine-tune an Execution Video Model (ExeVRM) that takes only a user instruction and a video-execution sequence to predict task success.
arXiv Detail & Related papers (2026-03-10T19:17:22Z) - VIPER: Process-aware Evaluation for Generative Video Reasoning [64.86465792516658]
We introduce VIPER, a comprehensive benchmark spanning 16 tasks across temporal, structural, symbolic, spatial, physics, and planning reasoning.<n>Our experiments reveal that state-of-the-art video models achieve only about 20% POC@1.0 and exhibit a significant outcome-hacking.
arXiv Detail & Related papers (2025-12-31T16:31:59Z) - TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Models [75.42002690128486]
TemporalBench is a new benchmark dedicated to evaluating fine-grained temporal understanding in videos.
It consists of 10K video question-answer pairs, derived from 2K high-quality human annotations detailing the temporal dynamics in video clips.
Results show that state-of-the-art models like GPT-4o achieve only 38.5% question answering accuracy on TemporalBench.
arXiv Detail & Related papers (2024-10-14T17:59:58Z) - Correspondence Matters for Video Referring Expression Comprehension [64.60046797561455]
Video Referring Expression (REC) aims to localize the referent objects described in the sentence to visual regions in the video frames.
Existing methods suffer from two problems: 1) inconsistent localization results across video frames; 2) confusion between the referent and contextual objects.
We propose a novel Dual Correspondence Network (dubbed as DCNet) which explicitly enhances the dense associations in both the inter-frame and cross-modal manners.
arXiv Detail & Related papers (2022-07-21T10:31:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.