NewtPhys: Do Foundation Models Understand Newtonian Physics?

• URL: http://arxiv.org/abs/2606.03986v1
• Date: Tue, 02 Jun 2026 17:59:12 GMT
• Title: NewtPhys: Do Foundation Models Understand Newtonian Physics?
• Abstract summary: Previous work has evaluated physics reasoning in foundation models using synthetic or semi-synthetic scenes.<n>NewtPhys is a 4D physically annotated dataset built from multiview images of real-world scenes with physics-grounded simulations.
• Score: 25.906063461629504
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Previous work has evaluated physics reasoning in foundation models using synthetic or semi-synthetic scenes and visual question-answering tasks. However, these benchmarks emphasize high-level events and lack the visual fidelity required to assess true low-level Newtonian understanding. We introduce NewtPhys, a 4D physically annotated dataset built from multiview images of real-world scenes with physics-grounded simulations. The dataset provides dense, fine-grained annotations across timesteps -- including 3D forces and amodal per-pixel quantities covering physics, tracking, semantics and geometry -- bridging the gap between simplistic synthetic setups and realistic visual complexity. Using NewtPhys, we systematically evaluate 56 VLMs, including 54 open-weight models and 2 closed-source frontier models, and 10 VFMs and reveal limitations in low-level physics reasoning. Beyond benchmarking, our dataset enables future research in physics-grounded vision and the development of next-generation physics-aware evaluations. Code and datasets are available at https://astra-vision.github.io/NewtPhys.

Related papers

• PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects [60.069451811298315]
  We introduce PhysX- Omni, a unified framework for simulation-ready physical 3D generation across diverse asset types.<n>Specifically, we develop a novel and efficient geometry representation tailored for Vision-Language Models, which directly encodes high-resolution 3D structures without compression.<n>In addition, we construct the first general simulation-ready 3D dataset, PhysXVerse, covering diverse indoor and outdoor categories.
  arXiv  Detail & Related papers  (2026-05-20T17:59:01Z)
• PhysInOne: Visual Physics Learning and Reasoning in One Suite [41.08902311182402]
  We present PhysInOne, a large-scale synthetic dataset addressing the critical scarcity of physically-grounded training data for AI systems.<n>Unlike existing datasets limited to merely hundreds or thousands of examples, PhysInOne provides 2 million videos across 153,810 dynamic 3D scenes.<n>We demonstrate PhysInOne's efficacy across four emerging applications: physics-aware video generation, long-/short-term future frame prediction, physical property estimation, and motion transfer.
  arXiv  Detail & Related papers  (2026-04-10T15:27:27Z)
• PhysAlign: Physics-Coherent Image-to-Video Generation through Feature and 3D Representation Alignment [23.066204478030627]
  Video Models (VDMs) offer a promising approach for simulating dynamic scenes and environments.<n>Existing models often generate temporally incoherent content that violates basic physical intuition.<n>We propose PhysAlign, an efficient framework for physics-coherent image-to-videoI (2V) generation.
  arXiv  Detail & Related papers  (2026-03-14T05:44:18Z)
• PhysicsMind: Sim and Real Mechanics Benchmarking for Physical Reasoning and Prediction in Foundational VLMs and World Models [40.16417939211015]
  Modern foundational Multimodal Large Language Models (MLLMs) and video world models have advanced significantly in mathematical, common-sense, and visual reasoning.<n>Existing benchmarks attempting to measure this matter rely on synthetic, Visual Question Answer templates or focus on perceptual video quality that is tangential to measuring how well the video abides by physical laws.<n>We introduce PhysicsMind, a unified benchmark that evaluates law-consistent reasoning and generation over three canonical principles: Center of Mass, Lever Equilibrium, and Newton's First Law.
  arXiv  Detail & Related papers  (2026-01-22T14:33:01Z)
• PhysWorld: From Real Videos to World Models of Deformable Objects via Physics-Aware Demonstration Synthesis [52.905353023326306]
  We propose PhysWorld, a framework that synthesizes physically plausible and diverse demonstrations to learn efficient world models.<n>Experiments show that PhysWorld has competitive performance while enabling inference speeds 47 times faster than the recent state-of-the-art method, i.e., PhysTwin.
  arXiv  Detail & Related papers  (2025-10-24T13:25:39Z)
• PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generation [53.06495362038348]
  Existing generation models excel at producing photo-realistic videos from text or images, but often lack physical plausibility and 3D controllability.<n>We introduce PhysCtrl, a novel framework for physics-grounded image-to-video generation with physical parameters and force control.<n> Experiments show that PhysCtrl generates realistic, physics-grounded motion trajectories which, when used to drive image-to-video models, yield high-fidelity, controllable videos.
  arXiv  Detail & Related papers  (2025-09-24T17:58:04Z)
• PhysGaia: A Physics-Aware Dataset of Multi-Body Interactions for Dynamic Novel View Synthesis [62.283499219361595]
  PhysGaia is a physics-aware dataset specifically designed for Dynamic Novel View Synthesis (DyNVS)<n>Our dataset provides complex dynamic scenarios with rich interactions among multiple objects.<n>PhysGaia will significantly advance research in dynamic view synthesis, physics-based scene understanding, and deep learning models integrated with physical simulation.
  arXiv  Detail & Related papers  (2025-06-03T12:19:18Z)
• Scaling Physical Reasoning with the PHYSICS Dataset [39.960500815534836]
  PHYSICS is a dataset containing 16,568 high-quality physics problems spanning subjects and difficulty levels.<n>It covers five major physics domains: Mechanics, Electromagnetism, Thermodynamics, Optics, and Modern Physics.<n>It also spans a wide range of difficulty levels, from high school to graduate-level physics courses.
  arXiv  Detail & Related papers  (2025-05-21T17:06:28Z)
• Latent Intuitive Physics: Learning to Transfer Hidden Physics from A 3D Video [58.043569985784806]
  We introduce latent intuitive physics, a transfer learning framework for physics simulation. It can infer hidden properties of fluids from a single 3D video and simulate the observed fluid in novel scenes. We validate our model in three ways: (i) novel scene simulation with the learned visual-world physics, (ii) future prediction of the observed fluid dynamics, and (iii) supervised particle simulation.
  arXiv  Detail & Related papers  (2024-06-18T16:37:44Z)
• Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Language [92.7638697243969]
  We propose a unified framework that can jointly learn visual concepts and infer physics models of objects from videos and language. This is achieved by seamlessly integrating three components: a visual perception module, a concept learner, and a differentiable physics engine.
  arXiv  Detail & Related papers  (2021-10-28T17:59:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.