GARDEN: Gravity-Aligned Reconstruction of Disentangled ENvironments from RGB images

• URL: http://arxiv.org/abs/2606.03921v1
• Date: Tue, 02 Jun 2026 17:13:01 GMT
• Title: GARDEN: Gravity-Aligned Reconstruction of Disentangled ENvironments from RGB images
• Abstract summary: We propose an RGB-only framework that reformulates reconstruction as physically-grounded scene factorization and outputs a structured hybrid scene representation.<n>The resulting representation combines explicit rigid bodies with a decoupled background, enabling direct physics simulation while preserving visual realism.
• Score: 61.2170105836525
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Converting multi-view RGB observations into simulation-ready 3D environments remains challenging because current reconstruction pipelines produce monolithic scene representations without explicit physical structure. They are typically defined up to an arbitrary global rotation and entangle rigid foreground objects with background geometry, which hinders stable physical interaction. Existing solutions often recover interactivity by replacing reconstructed objects with retrieved CAD assets, but this introduces a slow retrieval-and-replacement stage and weakens scene-specific geometric fidelity. We propose GARDEN, an RGB-only framework that reformulates reconstruction as physically-grounded scene factorization and outputs a structured hybrid scene representation. The key idea is to use gravity as a universal physical prior: we first align the reconstruction to a unified Gravity-View frame to resolve gauge ambiguity, then recover object-centric rigid meshes with accurate 6-DoF placement, and finally remove duplicate object geometry from the background through conditional 3D point classification. The resulting representation combines explicit rigid bodies with a decoupled background, enabling direct physics simulation while preserving visual realism. Experiments on both simulated and real multi-view scenes show that GARDEN improves object placement reliability, disentanglement quality, and rendering-simulation efficiency compared with retrieval-based baselines.

Related papers

• $φ$-Scene: Physically Grounded Image-to-3D Scene Reconstruction [48.26103456970799]
  $$-Scene is a physically grounded approach to open-vocabulary and compositional image-to-3D scene reconstruction.<n>It infers how objects support one another, orders them accordingly, and settles each object against its stabilized support context.<n>Experiments on 3D-Front show that $$-Scene achieves the strongest overall performance among out-of-domain methods.
  arXiv  Detail & Related papers  (2026-06-19T16:50:48Z)
• REST3D: Reconstructing Physically Stable 3D Scenes from a Single Image [31.061246129846044]
  Reconstructing physically stable 3D scenes from a single RGB image enables casual images to be converted into simulation-ready digital assets.<n>We propose REST3D, a single-image reconstruction framework that can reconstruct physically stable 3D scenes by integrating physical scene understanding with physics-constrained refinement.
  arXiv  Detail & Related papers  (2026-05-28T17:59:01Z)
• SAM3D-Phys: Towards Multi-Object Interactive Simulation in Real World [42.46212677643043]
  SAM3D-Phys is a framework that integrates scene reconstruction with generative 3D priors of SAM3D to recover physically simulatable objects.<n>By recovering complete object geometry and restoring its pose and appearance within the scene, SAM3D-Phys produces clean object representations suitable for physics-based simulation.
  arXiv  Detail & Related papers  (2026-05-28T17:07:51Z)
• Real-to-Sim for Highly Cluttered Environments via Physics-Consistent Inter-Object Reasoning [11.05191602561784]
  Reconstructing physically valid 3D scenes from single-view observations is a prerequisite for bridging the gap between visual perception and robotic control.<n>We propose a novel physics-constrained Real-to-Sim pipeline that reconstructs physically consistent 3D scenes from single-view RGB-D data.
  arXiv  Detail & Related papers  (2026-02-13T05:24:58Z)
• Joint Geometry-Appearance Human Reconstruction in a Unified Latent Space via Bridge Diffusion [57.09673862519791]
  This paper introduces textbfJGA-LBD, a novel framework that unifies the modeling of geometry and appearance into a joint latent representation.<n> Experiments demonstrate that JGA-LBD outperforms current state-of-the-art approaches in terms of both geometry fidelity and appearance quality.
  arXiv  Detail & Related papers  (2026-01-01T12:48:56Z)
• REArtGS: Reconstructing and Generating Articulated Objects via 3D Gaussian Splatting with Geometric and Motion Constraints [47.82928111264676]
  REArtGS is a novel framework that introduces additional geometric and motion constraints to 3D Gaussian primitives.<n>It achieves high-fidelity textured surface reconstruction for given states, and enables high-fidelity surface generation for unseen states.
  arXiv  Detail & Related papers  (2025-03-09T16:05:36Z)
• CAST: Component-Aligned 3D Scene Reconstruction from an RGB Image [44.8172828045897]
  Current methods often struggle with domain-specific limitations or low-quality object generation.<n>We propose CAST, a novel method for 3D scene reconstruction and recovery.
  arXiv  Detail & Related papers  (2025-02-18T14:29:52Z)
• Single-view 3D Mesh Reconstruction for Seen and Unseen Categories [69.29406107513621]
  Single-view 3D Mesh Reconstruction is a fundamental computer vision task that aims at recovering 3D shapes from single-view RGB images. This paper tackles Single-view 3D Mesh Reconstruction, to study the model generalization on unseen categories. We propose an end-to-end two-stage network, GenMesh, to break the category boundaries in reconstruction.
  arXiv  Detail & Related papers  (2022-08-04T14:13:35Z)
• From Points to Multi-Object 3D Reconstruction [71.17445805257196]
  We propose a method to detect and reconstruct multiple 3D objects from a single RGB image. A keypoint detector localizes objects as center points and directly predicts all object properties, including 9-DoF bounding boxes and 3D shapes. The presented approach performs lightweight reconstruction in a single-stage, it is real-time capable, fully differentiable and end-to-end trainable.
  arXiv  Detail & Related papers  (2020-12-21T18:52:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.