Prim2Room: Layout-Controllable Room Mesh Generation from Primitives

• URL: http://arxiv.org/abs/2409.05380v1
• Date: Mon, 9 Sep 2024 07:25:47 GMT
• Title: Prim2Room: Layout-Controllable Room Mesh Generation from Primitives
• Authors: Chengzeng Feng, Jiacheng Wei, Cheng Chen, Yang Li, Pan Ji, Fayao Liu, Hongdong Li, Guosheng Lin,
• Abstract summary: Prim2Room is a framework for controllable room mesh generation leveraging 2D layout conditions and 3D primitive retrieval. We introduce an adaptive viewpoint selection algorithm that allows the system to generate the furniture texture and geometry from more favorable views. Our method not only enhances the accuracy and aesthetic appeal of generated 3D scenes but also provides a user-friendly platform for detailed room design.
• Score: 90.5012354166981
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose Prim2Room, a novel framework for controllable room mesh generation leveraging 2D layout conditions and 3D primitive retrieval to facilitate precise 3D layout specification. Diverging from existing methods that lack control and precision, our approach allows for detailed customization of room-scale environments. To overcome the limitations of previous methods, we introduce an adaptive viewpoint selection algorithm that allows the system to generate the furniture texture and geometry from more favorable views than predefined camera trajectories. Additionally, we employ non-rigid depth registration to ensure alignment between generated objects and their corresponding primitive while allowing for shape variations to maintain diversity. Our method not only enhances the accuracy and aesthetic appeal of generated 3D scenes but also provides a user-friendly platform for detailed room design.

Related papers

• DECOLLAGE: 3D Detailization by Controllable, Localized, and Learned Geometry Enhancement [38.719572669042925]
  We present a 3D modeling method which enables end-users to refine or detailize 3D shapes using machine learning. We show that our ability to localize details enables novel interactive creative and applications.
  arXiv  Detail & Related papers  (2024-09-10T00:51:49Z)
• Text-guided Controllable Mesh Refinement for Interactive 3D Modeling [48.226234898333]
  We propose a novel technique for adding geometric details to an input coarse 3D mesh guided by a text prompt. First, we generate a single-view RGB image conditioned on the input coarse geometry and the input text prompt. Second, we use our novel multi-view normal generation architecture to jointly generate six different views of the normal images. Third, we optimize our mesh with respect to all views and generate a fine, detailed geometry as output.
  arXiv  Detail & Related papers  (2024-06-03T17:59:43Z)
• DreamControl: Control-Based Text-to-3D Generation with 3D Self-Prior [97.694840981611]
  We propose a two-stage 2D-lifting framework, namely DreamControl. It generates fine-grained objects with control-based score distillation. DreamControl can generate high-quality 3D content in terms of both geometry consistency and texture fidelity.
  arXiv  Detail & Related papers  (2023-12-11T15:12:50Z)
• ControlRoom3D: Room Generation using Semantic Proxy Rooms [48.93419701713694]
  We present ControlRoom3D, a novel method to generate high-quality room meshes. Our approach is a user-defined 3D semantic proxy room that outlines a rough room layout. When rendered to 2D, this 3D representation provides valuable geometric and semantic information to control powerful 2D models.
  arXiv  Detail & Related papers  (2023-12-08T17:55:44Z)
• Ctrl-Room: Controllable Text-to-3D Room Meshes Generation with Layout Constraints [35.073500525250346]
  We present Ctrl-Room, which can generate convincing 3D rooms with designer-style layouts and high-fidelity textures from just a text prompt. Ctrl-Room enables versatile interactive editing operations such as resizing or moving individual furniture items.
  arXiv  Detail & Related papers  (2023-10-05T15:29:52Z)
• DeformToon3D: Deformable 3D Toonification from Neural Radiance Fields [96.0858117473902]
  3D toonification involves transferring the style of an artistic domain onto a target 3D face with stylized geometry and texture. We propose DeformToon3D, an effective toonification framework tailored for hierarchical 3D GAN. Our approach decomposes 3D toonification into subproblems of geometry and texture stylization to better preserve the original latent space.
  arXiv  Detail & Related papers  (2023-09-08T16:17:45Z)
• MCTS with Refinement for Proposals Selection Games in Scene Understanding [32.92475660892122]
  We propose a novel method applicable in many scene understanding problems that adapts the Monte Carlo Tree Search (MCTS) algorithm. From a generated pool of proposals, our method jointly selects and optimize proposals that maximize the objective term. Our method shows high performance on the Matterport3D dataset without introducing hard constraints on room layout configurations.
  arXiv  Detail & Related papers  (2022-07-07T10:15:54Z)
• LED2-Net: Monocular 360 Layout Estimation via Differentiable Depth Rendering [59.63979143021241]
  We formulate the task of 360 layout estimation as a problem of predicting depth on the horizon line of a panorama. We propose the Differentiable Depth Rendering procedure to make the conversion from layout to depth prediction differentiable. Our method achieves state-of-the-art performance on numerous 360 layout benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-01T15:48:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.