RigidBench: Evaluating Rigid-Body Physics in Video Generation Models

• URL: http://arxiv.org/abs/2608.15555v1
• Date: Sun, 16 Aug 2026 06:05:20 GMT
• Title: RigidBench: Evaluating Rigid-Body Physics in Video Generation Models
• Abstract summary: We introduce a simulator-grounded benchmark that compares a generated continuation with a reference rollout from the same initial frame and motion description.<n>We evaluate eight models on the same 100 examples with ten measurements that keep these aspects separate.<n>Full fine-tuning reduces 3D trajectory error by about 20% with almost no change in SSIM.
• Score: 16.53417077740312
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video models are increasingly used to predict what happens next in a scene, yet the metrics commonly used to compare their outputs say little about whether the predicted objects move correctly. Motion, geometry, identity, background stability, and visual similarity can fail independently, but whole-frame scores often mix these errors together. We introduce RigidBench, a simulator-grounded benchmark that compares a generated continuation with a reference rollout from the same initial frame and motion description. Its five rigid-body tasks vary objects, materials, viewpoints, and indoor and outdoor scenes, with per-frame masks, depth, 6-DoF trajectories, and contacts available for scoring. We evaluate eight models on the same 100 examples with ten measurements that keep these aspects separate. The resulting rankings depend strongly on what is measured: no model leads on all ten, and across model means, higher SSIM accompanies larger 3D trajectory error (r = 0.89). RigidBench also includes 5,000 training videos with exact simulator state, which we use to fine-tune and analyze Wan 2.2 TI2V-5B. Full fine-tuning reduces 3D trajectory error by about 20% with almost no change in SSIM, while teacher-forced probes and targeted interventions show that object position is represented throughout Wan's diffusion transformer and used by its denoising computation.

Related papers

• RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstruction [24.674633972725804]
  We introduce RoboPhys-3D, a 3D-grounded embodied world model (EWM) benchmark built on RoboTwin 2.0.<n>A defining feature of RoboPhys-3D is that generated and ground-truth videos are processed through the same 3D reconstruction pipeline.<n>The benchmark organizes 50 complementary metrics into 18 sub-dimensions across four levels: pixel-level fidelity, 3D geometry consistency, state-level understanding, and task-level completeness.
  arXiv  Detail & Related papers  (2026-08-28T09:44:13Z)
• GAUGE: A Measurement-Grounded Benchmark for Physical Fidelity in Simulation Engines and Video World Models [66.77745044586173]
  GAUGE is a real-world-grounded diagnostic benchmark for evaluating how numerical simulators and generative video world models reproduce or deviate from real-world physics.<n>We benchmark Isaac Sim, Genesis, and Newton on 14 task families using generalized trajectory errors, and evaluate 6 image-to-video models on 5 rigid-body tasks.<n>Our results reveal no uniformly faithful physics engine, with the largest discrepancies arising in impulsive contact, rapid textile motion, and volumetric deformation.
  arXiv  Detail & Related papers  (2026-08-06T12:19:38Z)
• KineBench: Benchmarking Embodied World Models via IDM-Free Kinematic Grounding [58.541098216700796]
  evaluating the physical consistency of embodied world models is a critical open challenge.<n>Existing frameworks almost exclusively rely on Inverse Dynamics Models(IDMs) for action extraction.<n>We present KineBench, an IDM-free closed-loop benchmark for EWMs.
  arXiv  Detail & Related papers  (2026-07-22T08:04:17Z)
• PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation [37.9748770367306]
  Video generation models can produce physically implausible manipulations, including inconsistent robot-object interactions.<n>We propose PhysisForcing, a training framework that strengthens physical consistency by focusing supervision on physics-informative regions.<n>Our experiments show that PhysisForcing consistently improves embodied video generation over strong baselines.
  arXiv  Detail & Related papers  (2026-06-26T14:30:18Z)
• Pose6DAug: Physically Plausible Multi-view Object Swapping for Robot Data Augmentation [52.61098793949261]
  We introduce Pose6DAug, a failure-driven data augmentation framework.<n>We turn a policy's successful episodes into targeted demonstrations for its failure modes.<n>Our method operates directly in 3D, anchoring the target object with an explicit mesh driven by a temporally coherent 6D pose trajectory.
  arXiv  Detail & Related papers  (2026-06-18T11:41:25Z)
• What-If World: A Causal Benchmark for General World Models in Embodied Scenarios [23.527918480081013]
  Video generation models are increasingly used as world simulators for tasks like driving and robotic manipulation.<n>What matters in these settings is not whether a single video looks right, but whether the model's output changes when its input changes.<n>We test this by giving a model two prompts describing the same scene with one physical detail varied, and checking whether the two videos diverge the way physics predicts.
  arXiv  Detail & Related papers  (2026-05-26T19:02:26Z)
• Towards High-Consistency Embodied World Model with Multi-View Trajectory Videos [24.111891848073288]
  Embodied world models aim to predict and interact with the physical world through visual observations and actions.<n>MTV-World introduces Multi-view Trajectory-Video control for precise visuomotor prediction.<n>MTV-World achieves precise control execution and accurate physical interaction modeling in complex dual-arm scenarios.
  arXiv  Detail & Related papers  (2025-11-17T02:17:04Z)
• DO3D: Self-supervised Learning of Decomposed Object-aware 3D Motion and Depth from Monocular Videos [76.01906393673897]
  We propose a self-supervised method to jointly learn 3D motion and depth from monocular videos. Our system contains a depth estimation module to predict depth, and a new decomposed object-wise 3D motion (DO3D) estimation module to predict ego-motion and 3D object motion. Our model delivers superior performance in all evaluated settings.
  arXiv  Detail & Related papers  (2024-03-09T12:22:46Z)
• DeepSimHO: Stable Pose Estimation for Hand-Object Interaction via Physics Simulation [81.11585774044848]
  We present DeepSimHO, a novel deep-learning pipeline that combines forward physics simulation and backward gradient approximation with a neural network. Our method noticeably improves the stability of the estimation and achieves superior efficiency over test-time optimization.
  arXiv  Detail & Related papers  (2023-10-11T05:34:36Z)
• Learning to Segment Rigid Motions from Two Frames [72.14906744113125]
  We propose a modular network, motivated by a geometric analysis of what independent object motions can be recovered from an egomotion field. It takes two consecutive frames as input and predicts segmentation masks for the background and multiple rigidly moving objects, which are then parameterized by 3D rigid transformations. Our method achieves state-of-the-art performance for rigid motion segmentation on KITTI and Sintel.
  arXiv  Detail & Related papers  (2021-01-11T04:20:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.