Physics-IQ Verified

• URL: http://arxiv.org/abs/2606.18943v1
• Date: Wed, 17 Jun 2026 11:23:52 GMT
• Title: Physics-IQ Verified
• Abstract summary: We present a systematic audit of the Physics-IQ benchmark and expose shortcomings.<n>We propose three solutions that sharpen how we can measure physical understanding of video generative models.<n>Our resulting benchmark, Physics-IQ Verified, refines 57.6% of all samples and improves over 34.8% of prompts.
• Score: 58.94877959578469
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video generative models ( VGMs) have become a new frontier that can be used not just for video generation but for a multitude of downstream tasks, including world modeling. To advance these tasks, a good video model must understand the physical reality of the world. Evaluating this understanding is an emerging field and has led to the Physics-IQ benchmark, which quantifies this explicitly by comparing model-generated videos to real-world videos of physical experiments. In this work, we present a systematic audit of the Physics-IQ benchmark, expose shortcomings and propose three solutions that sharpen how we can measure physical understanding of VGMs. Specifically, we improve prompt and ground-truth quality to reduce the influence of confounding factors and further introduce a sample-level scoring system that weights each sample and metric equally. Our resulting benchmark, Physics-IQ Verified, refines 57.6\% of all samples and improves over 34.8\% of prompts. In a comparison study using six image-to-video generative models, we observe moderate but meaningful ranking changes (Kendall's $τ= 0.46$). We hope Physics-IQ Verified advances the community by providing a more reliable signal toward physically accurate VGMs. The code for the benchmark can be accessed at https://github.com/google-deepmind/physics-iq-benchmark

Related papers

• Apple-$π$: Benchmarking Thinking with Video Towards Law-Grounded Physical Intelligence [54.58886283991723]
  Apple-PI is the first benchmark that anchors video-model evaluation explicitly in physical laws.<n>Apple-PI comprises three components.<n> Benchmarking 11 models shows that current video models remain far from reliable law-grounded world simulators.
  arXiv  Detail & Related papers  (2026-07-17T18:00:05Z)
• Physics Question Scene Graph: Fine-grained Evaluation of Physical Plausibility in Text-to-Video Generation [72.04671111142191]
  We introduce Physics Question Scene Graph (PQSG), a hierarchical question-based evaluation pipeline.<n>PQSG evaluates generated videos by checking their faithfulness to a prompt across objects, actions, and adherence to physical laws.<n>We validate PQSG by creating FinePhyEval, a dataset with physics-based prompts and corresponding generated videos.
  arXiv  Detail & Related papers  (2026-06-24T02:12:54Z)
• Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generation [78.41956766048308]
  Proprio is a training-free framework that enables a frozen generator to assess and improve the physical plausibility of its own outputs.<n>Inspired by proprioception, the biological sense of one's own movement, Proprio treats the model's flow residual as a self-scoring signal.<n>Proprio consistently improves physical plausibility across text-to-video and image-to-video benchmarks.
  arXiv  Detail & Related papers  (2026-05-27T09:44:18Z)
• PhyGround: Benchmarking Physical Reasoning in Generative World Models [46.53245929361594]
  We introduce PhyGround, a criteria-grounded benchmark for evaluating physical reasoning in video generation.<n>The benchmark contains 250 curated prompts, each augmented with an expected physical outcome, and a taxonomy of 13 physical laws.<n>We evaluate eight modern video generation models through a large-scale, quality-controlled human study.
  arXiv  Detail & Related papers  (2026-05-11T16:30:51Z)
• PhysicsMind: Sim and Real Mechanics Benchmarking for Physical Reasoning and Prediction in Foundational VLMs and World Models [40.16417939211015]
  Modern foundational Multimodal Large Language Models (MLLMs) and video world models have advanced significantly in mathematical, common-sense, and visual reasoning.<n>Existing benchmarks attempting to measure this matter rely on synthetic, Visual Question Answer templates or focus on perceptual video quality that is tangential to measuring how well the video abides by physical laws.<n>We introduce PhysicsMind, a unified benchmark that evaluates law-consistent reasoning and generation over three canonical principles: Center of Mass, Lever Equilibrium, and Newton's First Law.
  arXiv  Detail & Related papers  (2026-01-22T14:33:01Z)
• PhysicsSolutionAgent: Towards Multimodal Explanations for Numerical Physics Problem Solving [1.2802720336459552]
  We introduce PhysicsSolutionAgent (PSA), an autonomous agent that generates physics-problem explanation videos.<n>We evaluate PSA on 32 videos spanning numerical and theoretical physics problems.<n>Using GPT-5-mini, PSA achieves a 100% video-completion rate with an average automated score of 3.8/5.
  arXiv  Detail & Related papers  (2026-01-19T23:11:02Z)
• PhyEduVideo: A Benchmark for Evaluating Text-to-Video Models for Physics Education [14.810845377459833]
  The benchmark is designed to assess how well T2V models can convey core physics concepts through visual illustrations.<n>Our aim is to systematically explore the feasibility of using T2V models to generate high-quality, curriculum-aligned educational content.
  arXiv  Detail & Related papers  (2026-01-02T18:42:02Z)
• Bootstrapping Physics-Grounded Video Generation through VLM-Guided Iterative Self-Refinement [51.54051161067026]
  We propose an iterative self-refinement framework to provide physics-aware guidance for video generation.<n>We introduce a multimodal chain-of-thought (MM-CoT) process that refines prompts based on feedback from physical inconsistencies.<n>Experiments on the PhyIQ benchmark show that our method improves the Physics-IQ score from 56.31 to 62.38.
  arXiv  Detail & Related papers  (2025-11-25T13:09:03Z)
• VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generation [66.58048825989239]
  VideoPhy-2 is an action-centric dataset for evaluating physical commonsense in generated videos.<n>We perform human evaluation that assesses semantic adherence, physical commonsense, and grounding of physical rules in the generated videos.<n>Our findings reveal major shortcomings, with even the best model achieving only 22% joint performance.
  arXiv  Detail & Related papers  (2025-03-09T22:49:12Z)
• Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation [51.750634349748736]
  Text-to-video (T2V) models have made significant strides in visualizing complex prompts. However, the capacity of these models to accurately represent intuitive physics remains largely unexplored. We introduce PhyGenBench to evaluate physical commonsense correctness in T2V generation.
  arXiv  Detail & Related papers  (2024-10-07T17:56:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.