$φ$-Scene: Physically Grounded Image-to-3D Scene Reconstruction

• URL: http://arxiv.org/abs/2606.21596v1
• Date: Fri, 19 Jun 2026 16:50:48 GMT
• Title: $φ$-Scene: Physically Grounded Image-to-3D Scene Reconstruction
• Abstract summary: $$-Scene is a physically grounded approach to open-vocabulary and compositional image-to-3D scene reconstruction.<n>It infers how objects support one another, orders them accordingly, and settles each object against its stabilized support context.<n>Experiments on 3D-Front show that $$-Scene achieves the strongest overall performance among out-of-domain methods.
• Score: 48.26103456970799
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reconstructing compositional 3D scenes from a single image is a fundamental challenge in 3D world modeling. Recent methods can recover high-fidelity, complete 3D objects and predict plausible scene arrangements, but most still treat scene reconstruction primarily as a visual and geometric prediction problem. Their outputs may therefore contain floating objects, interpenetrations, or unstable-contact artifacts, limiting their physical validity and downstream usability in simulation, robotics, and interactive environments. We present $φ$-Scene, a physically grounded approach to open-vocabulary and compositional image-to-3D scene reconstruction. The key premise is that a reconstructed scene should not be treated merely as a set of objects with predicted poses, but as a stable physical system. Accordingly, $φ$-Scene formulates reconstruction as topology-driven physical assembly: it infers how objects support one another, orders them accordingly, and progressively settles each object against its already stabilized support context. For each object in topological order, SDF-based optimization first resolves penetrations against the pre-settled support context, and rigid-body simulation then settles the object into a stable contact configuration under real-world physical constraints. Experiments on 3D-Front show that $φ$-Scene achieves the strongest overall performance among out-of-domain methods and remains highly competitive with in-domain baselines. Human and VLM evaluations further show strong preference for $φ$-Scene in visual quality, reference alignment, and physical plausibility. Finally, dedicated physical plausibility metrics covering static contact quality and dynamic stability demonstrate that $φ$-Scene substantially reduces penetration artifacts while producing much lower post-simulation drift, indicating more stable and physically grounded 3D scenes.

Related papers

• ReScene: Structured Indoor Scene Reconstruction from Multi-View Captures [18.468596508491967]
  ReScene is a framework that threads multi-view geometry throughout the pipeline as a unifying prior.<n>Our method consists of two main components: HierView prioritizes reconstruction views based on semantic consistency and 3D coverage.<n>ReScene sets a new state of the art across geometry, rendering, and perceptual quality on a set of ScanNet scenes.
  arXiv  Detail & Related papers  (2026-06-26T13:08:40Z)
• GARDEN: Gravity-Aligned Reconstruction of Disentangled ENvironments from RGB images [61.2170105836525]
  We propose an RGB-only framework that reformulates reconstruction as physically-grounded scene factorization and outputs a structured hybrid scene representation.<n>The resulting representation combines explicit rigid bodies with a decoupled background, enabling direct physics simulation while preserving visual realism.
  arXiv  Detail & Related papers  (2026-06-02T17:13:01Z)
• PhyScene3D: Physically Consistent Interactive 3D Tabletop Scene Generation [75.9028457575434]
  Generating 3D tabletop scenes is a fundamental yet underexplored problem for interactive and generalist robotic learning.<n>Here, we introduce PhyScene3D, a framework that reformulates generation as a Human-Mimetic Constructive Process.<n>Experiments demonstrate that PhyScene3D outperforms state-of-the-art approaches in both semantic accuracy and physical validity.
  arXiv  Detail & Related papers  (2026-06-01T03:56:32Z)
• REST3D: Reconstructing Physically Stable 3D Scenes from a Single Image [31.061246129846044]
  Reconstructing physically stable 3D scenes from a single RGB image enables casual images to be converted into simulation-ready digital assets.<n>We propose REST3D, a single-image reconstruction framework that can reconstruct physically stable 3D scenes by integrating physical scene understanding with physics-constrained refinement.
  arXiv  Detail & Related papers  (2026-05-28T17:59:01Z)
• SAM3D-Phys: Towards Multi-Object Interactive Simulation in Real World [42.46212677643043]
  SAM3D-Phys is a framework that integrates scene reconstruction with generative 3D priors of SAM3D to recover physically simulatable objects.<n>By recovering complete object geometry and restoring its pose and appearance within the scene, SAM3D-Phys produces clean object representations suitable for physics-based simulation.
  arXiv  Detail & Related papers  (2026-05-28T17:07:51Z)
• PhyScensis: Physics-Augmented LLM Agents for Complex Physical Scene Arrangement [89.35154754765502]
  PhyScensis is an agent-based framework powered by a physics engine to produce physically plausible scene configurations.<n>Our framework preserves strong controllability over fine-grained textual descriptions and numerical parameters.<n> Experimental results show that our method outperforms prior approaches in scene complexity, visual quality, and physical accuracy.
  arXiv  Detail & Related papers  (2026-02-16T17:55:25Z)
• PAT3D: Physics-Augmented Text-to-3D Scene Generation [47.18949891825537]
  PAT3D generates 3D objects, infers their spatial relations, and organizes them into a hierarchical scene tree.<n>A differentiable rigid-body simulator ensures realistic object interactions under gravity.<n>Experiments demonstrate that PAT3D substantially outperforms prior approaches in physical plausibility, semantic consistency, and visual quality.
  arXiv  Detail & Related papers  (2025-11-26T23:23:58Z)
• SCORP: Scene-Consistent Object Refinement via Proxy Generation and Tuning [46.441761732998536]
  We introduce Scene-Consistent Object Refinement via Proxy Generation and Tuning (SCORP)<n>SCORP is a novel 3D enhancement framework that leverages 3D generative priors to recover fine-grained object geometry and appearance under missing views.<n>It achieves consistent gains over recent state-of-the-art baselines on both novel view synthesis and geometry completion tasks.
  arXiv  Detail & Related papers  (2025-06-30T13:26:21Z)
• Physically Compatible 3D Object Modeling from a Single Image [109.98124149566927]
  We present a framework that transforms single images into 3D physical objects.<n>Our framework embeds physical compatibility into the reconstruction process.<n>It consistently enhances the physical realism of 3D models over existing methods.
  arXiv  Detail & Related papers  (2024-05-30T21:59:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.