NeoWorld-Pro: Programming Interactive Scenes from Monocular Images for Embodied Simulation
- URL: http://arxiv.org/abs/2608.24212v1
- Date: Tue, 25 Aug 2026 08:19:39 GMT
- Title: NeoWorld-Pro: Programming Interactive Scenes from Monocular Images for Embodied Simulation
- Abstract summary: NeoWorld-Pro is a framework that reformulates monocular scene reconstruction as procedural programming for interactive 3D environments.<n>A physics-in-the-loop mechanism then iteratively refines the generated programs by validating their execution in a physics engine.
- Score: 37.57654523550713
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The advancement of Embodied AI necessitates high-quality simulation assets that faithfully mirror the real world. However, transforming raw visual observations into simulation-ready scenes remains challenging due to the lack of physical grounding and scene-level interactivity in current image-to-URDF methods. We propose NeoWorld-Pro, a framework that reformulates monocular scene reconstruction as procedural programming for interactive 3D environments. Leveraging the zero-shot reasoning and code synthesis capabilities of MLLMs, NeoWorld-Pro converts a single RGB image into executable programs specifying object geometry, articulation, and physical properties. A physics-in-the-loop mechanism then iteratively refines the generated programs by validating their execution in a physics engine, enforcing physically plausible articulations, valid object compositions and interactions, and accurate spatial relationships. Experiments show that NeoWorld-Pro outperforms open-loop and prior monocular reconstruction methods, while enabling complex downstream tasks such as stable stacking and fine-grained manipulation.
Related papers
- SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation [50.175987640001644]
We present SimWorlds: a multi-agent framework that produces dynamic, editable 4D scenes from text.<n>We also introduce 4DBuildBench, a benchmark for assessing both visual fidelity and physical consistency of the procedural dynamic 3D scenes.
arXiv Detail & Related papers (2026-07-02T06:29:18Z) - ManiSplat: Manipulation Trajectory Synthesis from Monocular Video via Decoupled 3D Gaussian Splatting [28.256286705954846]
Reconstructing dynamic and interactive 3D scenes from real-world observations remains a fundamental challenge in computer vision and robotics.<n>We introduce ManiSplat, a unified framework that reconstructs controllable and decoupled Gaussian digital twins directly from robotic videos.<n>Our approach reconstructs interaction-driven dynamic scenes with high fidelity and controllability, effectively supporting downstream robotic tasks and policy learning.
arXiv Detail & Related papers (2026-06-09T09:55:58Z) - SceneCode: Executable World Programs for Editable Indoor Scenes with Articulated Objects [69.20984454755512]
Indoor scene synthesis underpins embodied AI, robotic manipulation, and simulation-based policy evaluation.<n>Existing pipelines represent generated content as static meshes and inherit articulation only from curated asset libraries.<n>We present SceneCode, a framework that compiles a natural language prompt into an executable, code-driven indoor world.
arXiv Detail & Related papers (2026-05-19T09:31:04Z) - TelePhysics: Physics-Grounded Multi-Object Scene Generation from a Single Image with Real-Time Interaction [51.01447538245441]
We propose TelePhysics, a training-free framework that converts a single image into a physically consistent and controllable video.<n>By representing the full scene geometry in a unified spatial coordinate system, TelePhysics resolves object penetration and alignment ambiguity.<n> Experimental results demonstrate that TelePhysics substantially outperforms prior methods in physical fidelity, spatial coherence, and controllability.
arXiv Detail & Related papers (2026-05-19T08:16:44Z) - HSImul3R: Physics-in-the-Loop Reconstruction of Simulation-Ready Human-Scene Interactions [69.43316445143408]
We present HSImul3R, a unified framework for simulation-ready 3D reconstruction of human-scene interactions.<n>Existing methods suffer from a perception-simulation gap, leading to instability in physics engines and failure in embodied AI applications.
arXiv Detail & Related papers (2026-03-16T17:58:33Z) - URDF-Anything+: Autoregressive Articulated 3D Models Generation for Physical Simulation [45.4820195450296]
We present an end-to-end autoregressive framework that directly generates executable articulated object models from visual observations.<n>Building on this capability, we enable a new Real-Follow-Sim paradigm, where high-fidelity digital twins constructed from visual observations allow policies trained in simulation to transfer to real robots without online adaptation.
arXiv Detail & Related papers (2026-03-14T16:23:44Z) - Mirage2Matter: A Physically Grounded Gaussian World Model from Video [87.9732484393686]
We present Simulate Anything, a graphics-driven world modeling and simulation framework.<n>Our approach reconstructs real-world environments into a photorealistic scene representation using 3D Gaussian Splatting (3DGS)<n>We then leverage generative models to recover a physically realistic representation and integrate it into a simulation environment via a precision calibration target.
arXiv Detail & Related papers (2026-01-24T07:43:57Z) - HoloScene: Simulation-Ready Interactive 3D Worlds from a Single Video [25.898073594115413]
HoloScene is a novel interactive 3D reconstruction framework.<n>It encodes object geometry, appearance, and physical properties alongside hierarchical and inter-object relationships.<n>The resulting digital twins exhibit complete and precise geometry, physical stability, and realistic rendering from novel viewpoints.
arXiv Detail & Related papers (2025-10-07T04:12:18Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.