TourPhysics: Bringing Physics to World Models for Exploration and Manipulation from a Single Image

• URL: http://arxiv.org/abs/2609.04911v2
• Date: Mon, 07 Sep 2026 04:56:06 GMT
• Title: TourPhysics: Bringing Physics to World Models for Exploration and Manipulation from a Single Image
• Abstract summary: TourPhysics is an online framework from a single image and a declarative physical configuration.<n>For each action, the simulator computes a finite physical and camera trajectory before the corresponding observation is generated.<n>TourPhysics follows prescribed camera and object trajectories more closely than the evaluated baselines.
• Score: 53.206752160427065
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Interactive visual world models must distinguish observation from physical intervention. Camera motion reveals new surfaces, whereas intervention changes object motion, contact, and deformation. Current video world models are largely driven by appearance priors and often lose physical or spatial consistency over long horizons. We present TourPhysics, an online framework initialized from a single image and a declarative physical configuration. TourPhysics extends PhysOmni, our ACM Multimedia 2026 work, from finite physics-grounded video synthesis to persistent exploration and manipulation. TourPhysics combines deterministic simulation with video generation while assigning separate roles to simulator state, geometric evidence, generator controls, and appearance memory. For each action, the simulator computes a finite physical and camera trajectory before the corresponding observation is generated. Accepted observations publish the terminal state and update the appearance memory and subsequent generator controls, while the committed state and simulator geometry remain fixed throughout synthesis and retry. We further separate the simulator geometry used for projection and visibility from the relative depth used to condition the generator. A reference-anchored memory retrieves accepted static appearance through geometric cross-view correspondence and incorporates it through a bounded residual that reverts to the native path when no valid correspondence exists. On simulator-defined camera tours and object manipulations, TourPhysics follows prescribed camera and object trajectories more closely than the evaluated baselines, preserves the input scene, and reduces appearance drift during long-horizon revisits.

Related papers

• GAUGE: A Measurement-Grounded Benchmark for Physical Fidelity in Simulation Engines and Video World Models [66.77745044586173]
  GAUGE is a real-world-grounded diagnostic benchmark for evaluating how numerical simulators and generative video world models reproduce or deviate from real-world physics.<n>We benchmark Isaac Sim, Genesis, and Newton on 14 task families using generalized trajectory errors, and evaluate 6 image-to-video models on 5 rigid-body tasks.<n>Our results reveal no uniformly faithful physics engine, with the largest discrepancies arising in impulsive contact, rapid textile motion, and volumetric deformation.
  arXiv  Detail & Related papers  (2026-08-06T12:19:38Z)
• PhysAgent: Reflective Agentic Physics Control for Physically Plausible Video Generation [68.13035350819901]
  PhysAgent is a framework that closes the loop among physical program generation, physics simulation, stage-specific verification, and targeted program repair.<n>Our framework produces more physically plausible videos, achieves better prompt alignment, and generalizes more effectively across diverse physical scenarios.
  arXiv  Detail & Related papers  (2026-07-17T08:55:09Z)
• SAM3D-Phys: Towards Multi-Object Interactive Simulation in Real World [42.46212677643043]
  SAM3D-Phys is a framework that integrates scene reconstruction with generative 3D priors of SAM3D to recover physically simulatable objects.<n>By recovering complete object geometry and restoring its pose and appearance within the scene, SAM3D-Phys produces clean object representations suitable for physics-based simulation.
  arXiv  Detail & Related papers  (2026-05-28T17:07:51Z)
• TelePhysics: Physics-Grounded Multi-Object Scene Generation from a Single Image with Real-Time Interaction [51.01447538245441]
  We propose TelePhysics, a training-free framework that converts a single image into a physically consistent and controllable video.<n>By representing the full scene geometry in a unified spatial coordinate system, TelePhysics resolves object penetration and alignment ambiguity.<n> Experimental results demonstrate that TelePhysics substantially outperforms prior methods in physical fidelity, spatial coherence, and controllability.
  arXiv  Detail & Related papers  (2026-05-19T08:16:44Z)
• PhyCo: Learning Controllable Physical Priors for Generative Motion [55.59209981836171]
  We present PhyCo, a framework that introduces continuous, interpretable, and physically grounded control into video generation.<n>Our approach integrates three key components: (i) a large-scale dataset of over 100K photorealistic simulation videos where friction, restitution, deformation, and force are systematically varied across diverse scenarios; (ii) physics-supervised fine-tuning of a pretrained diffusion model using a ControlNet conditioned on pixel-aligned physical property maps; and (iii) VLM-guided reward optimization, where a fine-tuned vision-language model evaluates generated videos with targeted physics queries and provides differentiable feedback.
  arXiv  Detail & Related papers  (2026-04-30T17:53:03Z)
• HSImul3R: Physics-in-the-Loop Reconstruction of Simulation-Ready Human-Scene Interactions [69.43316445143408]
  We present HSImul3R, a unified framework for simulation-ready 3D reconstruction of human-scene interactions.<n>Existing methods suffer from a perception-simulation gap, leading to instability in physics engines and failure in embodied AI applications.
  arXiv  Detail & Related papers  (2026-03-16T17:58:33Z)
• URDF-Anything+: Autoregressive Articulated 3D Models Generation for Physical Simulation [45.4820195450296]
  We present an end-to-end autoregressive framework that directly generates executable articulated object models from visual observations.<n>Building on this capability, we enable a new Real-Follow-Sim paradigm, where high-fidelity digital twins constructed from visual observations allow policies trained in simulation to transfer to real robots without online adaptation.
  arXiv  Detail & Related papers  (2026-03-14T16:23:44Z)
• PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videos [21.441062722848265]
  PhysTwin is a novel framework that uses sparse videos of dynamic objects under interaction to produce a photo- and physically realistic, real-time interactive replica.<n>Our approach centers on two key components: (1) a physics-informed representation that combines spring-mass models for realistic physical simulation, and generative shape models for geometry, and Gaussian splats for rendering.<n>Our method integrates an inverse physics framework with visual perception cues, enabling high-fidelity reconstruction even from partial, occluded, and limited viewpoints.
  arXiv  Detail & Related papers  (2025-03-23T07:49:19Z)
• Trajectory Optimization for Physics-Based Reconstruction of 3d Human Pose from Monocular Video [31.96672354594643]
  We focus on the task of estimating a physically plausible articulated human motion from monocular video. Existing approaches that do not consider physics often produce temporally inconsistent output with motion artifacts. We show that our approach achieves competitive results with respect to existing physics-based methods on the Human3.6M benchmark.
  arXiv  Detail & Related papers  (2022-05-24T18:02:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.