SceneReGen: Generative Reconstruction of 3D Scenes from a Single Image

• URL: http://arxiv.org/abs/2608.23930v1
• Date: Tue, 25 Aug 2026 00:29:37 GMT
• Title: SceneReGen: Generative Reconstruction of 3D Scenes from a Single Image
• Abstract summary: We introduce SceneReGen, a generative reconstruction framework that reinterprets scene reconstruction as the generation and assembly of complete object assets.<n>SceneReGen addresses the generation-reconstruction gap through selective pose factorization.<n>SceneReGen achieves the best scene-level CD, scene-level F-Score, and 3D bounding-box IoU among the evaluated methods.
• Score: 24.844593424101618
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Single-image 3D scene reconstruction must complete partially observed objects and place them coherently in a shared observation-aligned scene frame. Object-level generative priors offer strong completion ability, but their centered, scale-normalized outputs are typically expressed in an object frame, creating a fundamental representation gap between object generation and scene reconstruction. We introduce SceneReGen, a generative reconstruction framework that reinterprets scene reconstruction as the generation and assembly of complete object assets in a shared observation-aligned scene frame. SceneReGen addresses the generation-reconstruction gap through selective pose factorization: each object's observed orientation is encoded directly in the generated mesh, while translation and scale are estimated from instance-level and global scene evidence. Given a scene image and instance masks, a geometry encoder extracts dense cues; learnable shape queries condition a pretrained DiT-based 3D generator to produce complete meshes in their observed orientations, while position queries fuse object and scene features to assemble them in the shared frame. On the 3D-FUTURE evaluation subset, SceneReGen achieves the best scene-level CD, scene-level F-Score, and 3D bounding-box IoU among the evaluated methods, ties the best object-level CD, and ranks second in object-level F-Score. Qualitative outputs in autonomous-driving and embodied-AI scenes further illustrate the potential of asset-centric reconstruction beyond indoor furniture.

Related papers

• WorldSculpt: Generating Compositional Worlds from Grounded Videos [50.427204240556854]
  We study the problem of generating a compositional 3D representation of a cluttered scene containing hundreds of objects.<n>This task is challenging in densely cluttered scenes, where objects heavily occlude one another and each view reveals only a fraction of their geometry.<n>We show that complex scenes with hundreds of objects can instead be generated compositionally by adapting a strong single-object 3D generative prior to multi-view observations.
  arXiv  Detail & Related papers  (2026-09-04T17:59:35Z)
• Scenix: Sparse-View 3D Scene Reconstruction via Executable Scene Programs [36.87702450793552]
  We present textscScenix, a sparse-view 3D scene reconstruction framework via executable scene programs.<n>Given sparse views, textscScenix predicts executable scene programs through perception-grounded asset instantiation and closed-loop spatial refinement.<n>We further introduce observation-consistent supervision that aligns each target scene with the visual evidence available in its input views.
  arXiv  Detail & Related papers  (2026-08-07T09:24:56Z)
• ReScene: Structured Indoor Scene Reconstruction from Multi-View Captures [18.468596508491967]
  ReScene is a framework that threads multi-view geometry throughout the pipeline as a unifying prior.<n>Our method consists of two main components: HierView prioritizes reconstruction views based on semantic consistency and 3D coverage.<n>ReScene sets a new state of the art across geometry, rendering, and perceptual quality on a set of ScanNet scenes.
  arXiv  Detail & Related papers  (2026-06-26T13:08:40Z)
• SceneConductor: 3D Scene Generation from a Single Image with Multi-Agent Orchestration [32.39337008619354]
  We propose a multi-agent orchestration framework that decomposes single-image 3D scene generation into three structured stages.<n>We introduce a geometry-aware layout predictor supervised by sparse geometric priors derived from point maps.<n>Our method consistently outperforms prior approaches in geometric accuracy, spatial consistency, and perceptual realism.
  arXiv  Detail & Related papers  (2026-06-07T01:38:39Z)
• GenRecon: Bridging Generative Priors for Multi-View 3D Scene Reconstruction [69.35136600319714]
  We introduce a new approach to high-fidelity 3D scene reconstruction from multi-view RGB images.<n>We cast scene reconstruction as conditional 3D generation over a set of spatially-localized, overlapping chunks.<n>We obtain high-fidelity results that outperform cutting-edge reconstruction methods by 16%.
  arXiv  Detail & Related papers  (2026-05-22T17:49:59Z)
• DecoRec: Decomposed 3D Scene Reconstruction from Single-View Images via Object-Level Diffusion [65.65756111062005]
  textitDecoRec is a system designed to elevate single-view 2D images to a decomposed 3D scene mesh.<n>Our results demonstrate that DecoRec facilitates high-quality single-view scene reconstruction in both geometry and novel synthesis.
  arXiv  Detail & Related papers  (2026-05-16T04:23:48Z)
• OCH3R: Object-Centric Holistic 3D Reconstruction [36.45885767128656]
  We introduce OCH3R, a unified framework for Object-Centric Holistic 3D Reconstruction from a single RGB image.<n>OCH3R performs one forward pass to simultaneously predict all object instances with their 6D poses and detailed 3D reconstructions.<n>On standard indoor benchmarks, OCH3R achieves state-of-the-art performance across monocular depth estimation, open-vocabulary semantic segmentation, and RGB-only category-level 6D pose estimation.
  arXiv  Detail & Related papers  (2026-05-13T05:17:51Z)
• FurnSet: Exploiting Repeats for 3D Scene Reconstruction [3.948390930350905]
  Single-view 3D scene reconstruction involves inferring both object geometry and spatial layout.<n>Existing methods typically reconstruct objects independently or rely on implicit scene context.<n>We propose FurnSet, a framework that explicitly identifies and leverages repeated object instances to improve reconstruction.
  arXiv  Detail & Related papers  (2026-04-22T01:27:13Z)
• ObjectGS: Object-aware Scene Reconstruction and Scene Understanding via Gaussian Splatting [54.92763171355442]
  ObjectGS is an object-aware framework that unifies 3D scene reconstruction with semantic understanding.<n>We show through experiments that ObjectGS outperforms state-of-the-art methods on open-vocabulary and panoptic segmentation tasks.
  arXiv  Detail & Related papers  (2025-07-21T10:06:23Z)
• HiScene: Creating Hierarchical 3D Scenes with Isometric View Generation [50.206100327643284]
  HiScene is a novel hierarchical framework that bridges the gap between 2D image generation and 3D object generation.<n>We generate 3D content that aligns with 2D representations while maintaining compositional structure.
  arXiv  Detail & Related papers  (2025-04-17T16:33:39Z)
• CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graph Diffusion [83.30168660888913]
  We present CommonScenes, a fully generative model that converts scene graphs into corresponding controllable 3D scenes. Our pipeline consists of two branches, one predicting the overall scene layout via a variational auto-encoder and the other generating compatible shapes. The generated scenes can be manipulated by editing the input scene graph and sampling the noise in the diffusion model.
  arXiv  Detail & Related papers  (2023-05-25T17:39:13Z)
• Shelf-Supervised Mesh Prediction in the Wild [54.01373263260449]
  We propose a learning-based approach to infer 3D shape and pose of object from a single image. We first infer a volumetric representation in a canonical frame, along with the camera pose. The coarse volumetric prediction is then converted to a mesh-based representation, which is further refined in the predicted camera frame.
  arXiv  Detail & Related papers  (2021-02-11T18:57:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.