SimuScene: Simulation-Ready Compositional 3D Scene Reconstruction from a Single Image

• URL: http://arxiv.org/abs/2606.03994v1
• Date: Tue, 02 Jun 2026 17:59:59 GMT
• Title: SimuScene: Simulation-Ready Compositional 3D Scene Reconstruction from a Single Image
• Abstract summary: We introduce SimuScene, a compositional 3D reconstruction pipeline that puts physics in the loop of shape and layout estimation.<n>By diagnostically simulating reconstructed objects under gravity, we convert penetration and support failures into quantitative correction signals.<n>This physics-informed feedback loop mitigates accumulated reconstruction errors and produces a stable, simulation-ready scene.
• Score: 18.870075620011615
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reconstructing interactive, simulation-ready 3D scenes from a single image is a critical bottleneck for robotic manipulation. While recent single-image lifters recover plausible per-object shapes, composing them yields scenes that collapse under physical simulation due to interpenetrating, hovering, or sinking objects. Existing physics-aware methods address this strictly as a post-hoc layout correction, leaving the underlying geometric errors unresolved. To address this, we introduce SimuScene, a compositional 3D reconstruction pipeline that puts physics in the loop of shape and layout estimation. Rather than using physics merely for layout cleanup, we utilize the physics engine as a diagnostic measurement tool during the generative process itself. By diagnostically simulating reconstructed objects under gravity, we convert penetration and support failures into quantitative correction signals that drive gravity-axis stretching and amodal shape resampling. This physics-informed feedback loop mitigates accumulated reconstruction errors and produces a stable, simulation-ready compositional 3D scene. Extensive experiments demonstrate state-of-the-art performance on physical stability and geometric alignment benchmarks. We further highlight SimuScene's utility by deploying reconstructed environments in humanoid control and robot-arm manipulation tasks.

Related papers

• TourPhysics: Bringing Physics to World Models for Exploration and Manipulation from a Single Image [53.206752160427065]
  TourPhysics is an online framework from a single image and a declarative physical configuration.<n>For each action, the simulator computes a finite physical and camera trajectory before the corresponding observation is generated.<n>TourPhysics follows prescribed camera and object trajectories more closely than the evaluated baselines.
  arXiv  Detail & Related papers  (2026-09-04T09:15:03Z)
• PhysMirror: Physics-Aware Mirror Object Generation [27.32616510065338]
  We introduce a novel, end-to-end physics-aware generation framework namely PhysMirror.<n>Our method automatically lifts prompted objects into 3D meshes and constructs a lightweight, mathematically exact mirror scene.<n>We also introduce Mirror Consistency Score (MCS), reference-free, fully automated metric that quantifies physical correctness.
  arXiv  Detail & Related papers  (2026-07-03T16:32:02Z)
• GARDEN: Gravity-Aligned Reconstruction of Disentangled ENvironments from RGB images [61.2170105836525]
  We propose an RGB-only framework that reformulates reconstruction as physically-grounded scene factorization and outputs a structured hybrid scene representation.<n>The resulting representation combines explicit rigid bodies with a decoupled background, enabling direct physics simulation while preserving visual realism.
  arXiv  Detail & Related papers  (2026-06-02T17:13:01Z)
• REST3D: Reconstructing Physically Stable 3D Scenes from a Single Image [31.061246129846044]
  Reconstructing physically stable 3D scenes from a single RGB image enables casual images to be converted into simulation-ready digital assets.<n>We propose REST3D, a single-image reconstruction framework that can reconstruct physically stable 3D scenes by integrating physical scene understanding with physics-constrained refinement.
  arXiv  Detail & Related papers  (2026-05-28T17:59:01Z)
• SAM3D-Phys: Towards Multi-Object Interactive Simulation in Real World [42.46212677643043]
  SAM3D-Phys is a framework that integrates scene reconstruction with generative 3D priors of SAM3D to recover physically simulatable objects.<n>By recovering complete object geometry and restoring its pose and appearance within the scene, SAM3D-Phys produces clean object representations suitable for physics-based simulation.
  arXiv  Detail & Related papers  (2026-05-28T17:07:51Z)
• TelePhysics: Physics-Grounded Multi-Object Scene Generation from a Single Image with Real-Time Interaction [51.01447538245441]
  We propose TelePhysics, a training-free framework that converts a single image into a physically consistent and controllable video.<n>By representing the full scene geometry in a unified spatial coordinate system, TelePhysics resolves object penetration and alignment ambiguity.<n> Experimental results demonstrate that TelePhysics substantially outperforms prior methods in physical fidelity, spatial coherence, and controllability.
  arXiv  Detail & Related papers  (2026-05-19T08:16:44Z)
• HSImul3R: Physics-in-the-Loop Reconstruction of Simulation-Ready Human-Scene Interactions [69.43316445143408]
  We present HSImul3R, a unified framework for simulation-ready 3D reconstruction of human-scene interactions.<n>Existing methods suffer from a perception-simulation gap, leading to instability in physics engines and failure in embodied AI applications.
  arXiv  Detail & Related papers  (2026-03-16T17:58:33Z)
• PAT3D: Physics-Augmented Text-to-3D Scene Generation [47.18949891825537]
  PAT3D generates 3D objects, infers their spatial relations, and organizes them into a hierarchical scene tree.<n>A differentiable rigid-body simulator ensures realistic object interactions under gravity.<n>Experiments demonstrate that PAT3D substantially outperforms prior approaches in physical plausibility, semantic consistency, and visual quality.
  arXiv  Detail & Related papers  (2025-11-26T23:23:58Z)
• Dynamic Scene 3D Reconstruction of an Uncooperative Resident Space Object [0.0]
  This study evaluates the performance of existing state-of-the-art 3D reconstruction algorithms for dynamic scenes.<n>Preliminary results on static scenes using Neuralangelo demonstrate promising reconstruction quality.
  arXiv  Detail & Related papers  (2025-09-09T17:16:38Z)
• Is Single-View Mesh Reconstruction Ready for Robotics? [78.14584238127338]
  We evaluate single-view mesh reconstruction models for their potential in enabling instant digital twin creation for real-time planning and dynamics prediction using physics simulators for robotic manipulation.<n>Our findings highlight critical gaps between computer vision advances and robotics needs, guiding future research at this intersection.
  arXiv  Detail & Related papers  (2025-05-23T14:35:56Z)
• DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors [75.83647027123119]
  We propose to learn the physical properties of a material field with video diffusion priors.<n>We then utilize a physics-based Material-Point-Method simulator to generate 4D content with realistic motions.
  arXiv  Detail & Related papers  (2024-06-03T16:05:25Z)
• GeoSim: Photorealistic Image Simulation with Geometry-Aware Composition [81.24107630746508]
  We present GeoSim, a geometry-aware image composition process that synthesizes novel urban driving scenes. We first build a diverse bank of 3D objects with both realistic geometry and appearance from sensor data. The resulting synthetic images are photorealistic, traffic-aware, and geometrically consistent, allowing image simulation to scale to complex use cases.
  arXiv  Detail & Related papers  (2021-01-16T23:00:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.