Benchmarking Scientific Understanding and Reasoning for Video Generation using VideoScience-Bench

• URL: http://arxiv.org/abs/2512.02942v1
• Date: Tue, 02 Dec 2025 17:11:23 GMT
• Title: Benchmarking Scientific Understanding and Reasoning for Video Generation using VideoScience-Bench
• Abstract summary: VideoScience-Bench is a benchmark designed to evaluate undergraduate-level scientific understanding in video models.<n>The benchmark comprises 200 carefully curated prompts spanning 14 topics and 103 concepts in physics and chemistry.<n>We conduct expert-annotated evaluations across seven state-of-the-art video models in T2V and I2V settings along five dimensions.
• Score: 19.261278000675773
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The next frontier for video generation lies in developing models capable of zero-shot reasoning, where understanding real-world scientific laws is crucial for accurate physical outcome modeling under diverse conditions. However, existing video benchmarks are physical commonsense-based, offering limited insight into video models' scientific reasoning capability. We introduce VideoScience-Bench, a benchmark designed to evaluate undergraduate-level scientific understanding in video models. Each prompt encodes a composite scientific scenario that requires understanding and reasoning across multiple scientific concepts to generate the correct phenomenon. The benchmark comprises 200 carefully curated prompts spanning 14 topics and 103 concepts in physics and chemistry. We conduct expert-annotated evaluations across seven state-of-the-art video models in T2V and I2V settings along five dimensions: Prompt Consistency, Phenomenon Congruency, Correct Dynamism, Immutability, and Spatio-Temporal Continuity. Using a VLM-as-a-Judge to assess video generations, we observe strong correlation with human assessments. To the best of our knowledge, VideoScience-Bench is the first benchmark to evaluate video models not only as generators but also as reasoners, requiring their generations to demonstrate scientific understanding consistent with expected physical and chemical phenomena. Our data and evaluation code are available at: \href{https://github.com/hao-ai-lab/VideoScience}{github.com/hao-ai-lab/VideoScience}.

Related papers

• Sci-VBench: Evaluating Knowledge- and Reasoning-Intensive Video Generation in Science Domains [15.749662582403614]
  We introduce Sci-VBench, a benchmark for evaluating knowledge- and reasoning-intensive video generation across scientific domains.<n>It contains 1,253 expert-annotated examples spanning 60 subjects across four core disciplines: Natural Science, Healthcare, Humanities & Social Sciences, and Engineering.
  arXiv  Detail & Related papers  (2026-08-10T17:27:32Z)
• Physion-Eval: Evaluating Physical Realism in Generated Video via Human Reasoning [77.34919361116037]
  Video generation models are increasingly used as world simulators for storytelling, simulation, and embodied AI.<n>Existing evaluations largely rely on automated metrics or coarse human judgments such as preferences or rubric-based checks.<n>We introduce Physion-Eval, a benchmark of expert human reasoning for diagnosing physical realism failures in videos generated by five state-of-the-art models.
  arXiv  Detail & Related papers  (2026-03-20T03:25:41Z)
• PhysicsMind: Sim and Real Mechanics Benchmarking for Physical Reasoning and Prediction in Foundational VLMs and World Models [40.16417939211015]
  Modern foundational Multimodal Large Language Models (MLLMs) and video world models have advanced significantly in mathematical, common-sense, and visual reasoning.<n>Existing benchmarks attempting to measure this matter rely on synthetic, Visual Question Answer templates or focus on perceptual video quality that is tangential to measuring how well the video abides by physical laws.<n>We introduce PhysicsMind, a unified benchmark that evaluates law-consistent reasoning and generation over three canonical principles: Center of Mass, Lever Equilibrium, and Newton's First Law.
  arXiv  Detail & Related papers  (2026-01-22T14:33:01Z)
• PhyEduVideo: A Benchmark for Evaluating Text-to-Video Models for Physics Education [14.810845377459833]
  The benchmark is designed to assess how well T2V models can convey core physics concepts through visual illustrations.<n>Our aim is to systematically explore the feasibility of using T2V models to generate high-quality, curriculum-aligned educational content.
  arXiv  Detail & Related papers  (2026-01-02T18:42:02Z)
• MoReGen: Multi-Agent Motion-Reasoning Engine for Code-based Text-to-Video Synthesis [20.319439629468263]
  We study Newtonian motion-controlled text-to-video generation and evaluation, emphasizing physical precision and motion coherence.<n>We introduce MoReGen, a motion-aware, physics-grounded T2V framework that generates physically accurate videos from text prompts in the code domain.<n>Our results reveal that state-of-the-art models struggle to maintain physical validity, while MoReGen establishes a direction toward physically coherent video synthesis.
  arXiv  Detail & Related papers  (2025-12-03T19:44:04Z)
• SciVideoBench: Benchmarking Scientific Video Reasoning in Large Multimodal Models [89.10286051587151]
  We introduce SciVideoBench, a rigorous benchmark designed to assess advanced video reasoning in scientific contexts.<n>SciVideoBench consists of 1,000 carefully crafted multiple-choice questions derived from cutting-edge scientific experimental videos.<n>Our evaluation highlights significant performance deficits in state-of-the-art proprietary and open-source LMMs.
  arXiv  Detail & Related papers  (2025-10-09T17:59:23Z)
• VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Models [53.204403109208506]
  Current text-to-video (T2V) models often struggle to generate physically plausible content.<n>We propose VideoREPA, which distills physics understanding capability from understanding foundation models into T2V models.
  arXiv  Detail & Related papers  (2025-05-29T17:06:44Z)
• VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understanding [70.00000053847738]
  Real visual understanding is essential for AI systems that interact with the physical world.<n>Current evaluations mostly use real-world videos similar to training data.<n>We propose negative-control tests using videos that depict physically impossible or logically inconsistent events.
  arXiv  Detail & Related papers  (2025-05-02T15:58:38Z)
• Morpheus: Benchmarking Physical Reasoning of Video Generative Models with Real Physical Experiments [55.56181322973467]
  We introduce Morpheus, a benchmark for evaluating video generation models on physical reasoning.<n>It features 80 real-world videos capturing physical phenomena, guided by conservation laws.<n>Our findings reveal that even with advanced prompting and video conditioning, current models struggle to encode physical principles.
  arXiv  Detail & Related papers  (2025-04-03T15:21:17Z)
• VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generation [66.58048825989239]
  VideoPhy-2 is an action-centric dataset for evaluating physical commonsense in generated videos.<n>We perform human evaluation that assesses semantic adherence, physical commonsense, and grounding of physical rules in the generated videos.<n>Our findings reveal major shortcomings, with even the best model achieving only 22% joint performance.
  arXiv  Detail & Related papers  (2025-03-09T22:49:12Z)
• Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation [51.750634349748736]
  Text-to-video (T2V) models have made significant strides in visualizing complex prompts. However, the capacity of these models to accurately represent intuitive physics remains largely unexplored. We introduce PhyGenBench to evaluate physical commonsense correctness in T2V generation.
  arXiv  Detail & Related papers  (2024-10-07T17:56:04Z)
• VideoPhy: Evaluating Physical Commonsense for Video Generation [93.28748850301949]
  We present VideoPhy, a benchmark designed to assess whether the generated videos follow physical commonsense for real-world activities. We then generate videos conditioned on captions from diverse state-of-the-art text-to-video generative models. Our human evaluation reveals that the existing models severely lack the ability to generate videos adhering to the given text prompts.
  arXiv  Detail & Related papers  (2024-06-05T17:53:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.