Prim2Room: Layout-Controllable Room Mesh Generation from Primitives

• URL: http://arxiv.org/abs/2409.05380v1
• Date: Mon, 9 Sep 2024 07:25:47 GMT
• Title: Prim2Room: Layout-Controllable Room Mesh Generation from Primitives
• Authors: Chengzeng Feng, Jiacheng Wei, Cheng Chen, Yang Li, Pan Ji, Fayao Liu, Hongdong Li, Guosheng Lin,
• Abstract summary: Prim2Room is a framework for controllable room mesh generation leveraging 2D layout conditions and 3D primitive retrieval. We introduce an adaptive viewpoint selection algorithm that allows the system to generate the furniture texture and geometry from more favorable views. Our method not only enhances the accuracy and aesthetic appeal of generated 3D scenes but also provides a user-friendly platform for detailed room design.
• Score: 90.5012354166981
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose Prim2Room, a novel framework for controllable room mesh generation leveraging 2D layout conditions and 3D primitive retrieval to facilitate precise 3D layout specification. Diverging from existing methods that lack control and precision, our approach allows for detailed customization of room-scale environments. To overcome the limitations of previous methods, we introduce an adaptive viewpoint selection algorithm that allows the system to generate the furniture texture and geometry from more favorable views than predefined camera trajectories. Additionally, we employ non-rigid depth registration to ensure alignment between generated objects and their corresponding primitive while allowing for shape variations to maintain diversity. Our method not only enhances the accuracy and aesthetic appeal of generated 3D scenes but also provides a user-friendly platform for detailed room design.

Related papers

• LinPrim: Linear Primitives for Differentiable Volumetric Rendering [53.780682194322225]
  We introduce two new scene representations based on linear primitives-octahedra and tetrahedra-both of which define homogeneous volumes bounded by triangular faces. This formulation aligns naturally with standard mesh-based tools, minimizing overhead for downstream applications. We demonstrate comparable performance to state-of-the-art volumetric methods while requiring fewer primitives to achieve similar reconstruction fidelity.
  arXiv  Detail & Related papers  (2025-01-27T18:49:38Z)
• Layout2Scene: 3D Semantic Layout Guided Scene Generation via Geometry and Appearance Diffusion Priors [52.63385546943866]
  We present a text-to-scene generation method (namely, Layout2Scene) using additional semantic layout as the prompt to inject precise control of 3D object positions. To fully leverage 2D diffusion priors in geometry and appearance generation, we introduce a semantic-guided geometry diffusion model and a semantic-geometry guided diffusion model. Our method can generate more plausible and realistic scenes as compared to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2025-01-05T12:20:13Z)
• Architect: Generating Vivid and Interactive 3D Scenes with Hierarchical 2D Inpainting [47.014044892025346]
  Architect is a generative framework that creates complex and realistic 3D embodied environments leveraging diffusion-based 2D image inpainting. Our pipeline is further extended to a hierarchical and iterative inpainting process to continuously generate placement of large furniture and small objects to enrich the scene.
  arXiv  Detail & Related papers  (2024-11-14T22:15:48Z)
• DECOLLAGE: 3D Detailization by Controllable, Localized, and Learned Geometry Enhancement [38.719572669042925]
  We present a 3D modeling method which enables end-users to refine or detailize 3D shapes using machine learning. We show that our ability to localize details enables novel interactive creative and applications.
  arXiv  Detail & Related papers  (2024-09-10T00:51:49Z)
• DreamControl: Control-Based Text-to-3D Generation with 3D Self-Prior [97.694840981611]
  We propose a two-stage 2D-lifting framework, namely DreamControl. It generates fine-grained objects with control-based score distillation. DreamControl can generate high-quality 3D content in terms of both geometry consistency and texture fidelity.
  arXiv  Detail & Related papers  (2023-12-11T15:12:50Z)
• ControlRoom3D: Room Generation using Semantic Proxy Rooms [48.93419701713694]
  We present ControlRoom3D, a novel method to generate high-quality room meshes. Our approach is a user-defined 3D semantic proxy room that outlines a rough room layout. When rendered to 2D, this 3D representation provides valuable geometric and semantic information to control powerful 2D models.
  arXiv  Detail & Related papers  (2023-12-08T17:55:44Z)
• MCTS with Refinement for Proposals Selection Games in Scene Understanding [32.92475660892122]
  We propose a novel method applicable in many scene understanding problems that adapts the Monte Carlo Tree Search (MCTS) algorithm. From a generated pool of proposals, our method jointly selects and optimize proposals that maximize the objective term. Our method shows high performance on the Matterport3D dataset without introducing hard constraints on room layout configurations.
  arXiv  Detail & Related papers  (2022-07-07T10:15:54Z)
• LED2-Net: Monocular 360 Layout Estimation via Differentiable Depth Rendering [59.63979143021241]
  We formulate the task of 360 layout estimation as a problem of predicting depth on the horizon line of a panorama. We propose the Differentiable Depth Rendering procedure to make the conversion from layout to depth prediction differentiable. Our method achieves state-of-the-art performance on numerous 360 layout benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-01T15:48:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.