RoomDesigner: Encoding Anchor-latents for Style-consistent and Shape-compatible Indoor Scene Generation

• URL: http://arxiv.org/abs/2310.10027v1
• Date: Mon, 16 Oct 2023 03:05:19 GMT
• Title: RoomDesigner: Encoding Anchor-latents for Style-consistent and Shape-compatible Indoor Scene Generation
• Authors: Yiqun Zhao, Zibo Zhao, Jing Li, Sixun Dong, Shenghua Gao
• Abstract summary: Indoor scene generation aims at creating shape-compatible, style-consistent furniture arrangements within a spatially reasonable layout. We propose a two-stage model integrating shape priors into the indoor scene generation by encoding furniture as anchor latent representations.
• Score: 26.906174238830474
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Indoor scene generation aims at creating shape-compatible, style-consistent furniture arrangements within a spatially reasonable layout. However, most existing approaches primarily focus on generating plausible furniture layouts without incorporating specific details related to individual furniture pieces. To address this limitation, we propose a two-stage model integrating shape priors into the indoor scene generation by encoding furniture as anchor latent representations. In the first stage, we employ discrete vector quantization to encode furniture pieces as anchor-latents. Based on the anchor-latents representation, the shape and location information of the furniture was characterized by a concatenation of location, size, orientation, class, and our anchor latent. In the second stage, we leverage a transformer model to predict indoor scenes autoregressively. Thanks to incorporating the proposed anchor-latents representations, our generative model produces shape-compatible and style-consistent furniture arrangements and synthesis furniture in diverse shapes. Furthermore, our method facilitates various human interaction applications, such as style-consistent scene completion, object mismatch correction, and controllable object-level editing. Experimental results on the 3D-Front dataset demonstrate that our approach can generate more consistent and compatible indoor scenes compared to existing methods, even without shape retrieval. Additionally, extensive ablation studies confirm the effectiveness of our design choices in the indoor scene generation model.

Related papers

• Prim2Room: Layout-Controllable Room Mesh Generation from Primitives [90.5012354166981]
  Prim2Room is a framework for controllable room mesh generation leveraging 2D layout conditions and 3D primitive retrieval. We introduce an adaptive viewpoint selection algorithm that allows the system to generate the furniture texture and geometry from more favorable views. Our method not only enhances the accuracy and aesthetic appeal of generated 3D scenes but also provides a user-friendly platform for detailed room design.
  arXiv  Detail & Related papers  (2024-09-09T07:25:47Z)
• Mixed Diffusion for 3D Indoor Scene Synthesis [55.94569112629208]
  We present MiDiffusion, a novel mixed discrete-continuous diffusion model architecture. We represent a scene layout by a 2D floor plan and a set of objects, each defined by its category, location, size, and orientation. Our experimental results demonstrate that MiDiffusion substantially outperforms state-of-the-art autoregressive and diffusion models in floor-conditioned 3D scene synthesis.
  arXiv  Detail & Related papers  (2024-05-31T17:54:52Z)
• Style-Consistent 3D Indoor Scene Synthesis with Decoupled Objects [84.45345829270626]
  Controllable 3D indoor scene synthesis stands at the forefront of technological progress. Current methods for scene stylization are limited to applying styles to the entire scene. We introduce a unique pipeline designed for synthesis 3D indoor scenes.
  arXiv  Detail & Related papers  (2024-01-24T03:10:36Z)
• FurniScene: A Large-scale 3D Room Dataset with Intricate Furnishing Scenes [57.47534091528937]
  FurniScene is a large-scale 3D room dataset with intricate furnishing scenes from interior design professionals. Specifically, the FurniScene consists of 11,698 rooms and 39,691 unique furniture CAD models with 89 different types. To better suit fine-grained indoor scene layout generation, we introduce a novel Two-Stage Diffusion Scene Model (TSDSM)
  arXiv  Detail & Related papers  (2024-01-07T12:34:45Z)
• iBARLE: imBalance-Aware Room Layout Estimation [54.819085005591894]
  Room layout estimation predicts layouts from a single panorama. There are significant imbalances in real-world datasets including the dimensions of layout complexity, camera locations, and variation in scene appearance. We propose imBalance-Aware Room Layout Estimation (iBARLE) framework to address these issues. iBARLE consists of (1) Appearance Variation Generation (AVG) module, (2) Complex Structure Mix-up (CSMix) module, which enhances generalizability w.r.t. room structure, and (3) a gradient-based layout objective function.
  arXiv  Detail & Related papers  (2023-08-29T06:20:36Z)
• CLIP-Layout: Style-Consistent Indoor Scene Synthesis with Semantic Furniture Embedding [17.053844262654223]
  Indoor scene synthesis involves automatically picking and placing furniture appropriately on a floor plan. This paper introduces an auto-regressive scene model which can output instance-level predictions. Our model achieves SOTA results in scene synthesis and improves auto-completion metrics by over 50%.
  arXiv  Detail & Related papers  (2023-03-07T00:26:02Z)
• Structured Graph Variational Autoencoders for Indoor Furniture layout Generation [7.035614458419328]
  We present a structured graph variational autoencoder for generating the layout of indoor 3D scenes. The architecture consists of a graph encoder that maps the input graph to a structured latent space, and a graph decoder that generates a furniture graph. Experiments on the 3D-FRONT dataset show that our method produces scenes that are diverse and are adapted to the room layout.
  arXiv  Detail & Related papers  (2022-04-11T04:58:26Z)
• Compositional Transformers for Scene Generation [13.633811200719627]
  We introduce the GANformer2 model, an iterative object-oriented transformer, explored for the task of generative modeling. We show it achieves state-of-the-art performance in terms of visual quality, diversity and consistency. Further experiments demonstrate the model's disentanglement and provide a deeper insight into its generative process.
  arXiv  Detail & Related papers  (2021-11-17T08:11:42Z)
• ATISS: Autoregressive Transformers for Indoor Scene Synthesis [112.63708524926689]
  We present ATISS, a novel autoregressive transformer architecture for creating synthetic indoor environments. We argue that this formulation is more natural, as it makes ATISS generally useful beyond fully automatic room layout synthesis. Our model is trained end-to-end as an autoregressive generative model using only labeled 3D bounding boxes as supervision.
  arXiv  Detail & Related papers  (2021-10-07T17:58:05Z)
• Image-Driven Furniture Style for Interactive 3D Scene Modeling [8.8561720398658]
  Interior style follows rules involving color, geometry and other visual elements. We propose a method for fast-tracking style-similarity tasks, by learning a furniture's style-compatibility from interior scene images. We demonstrate our method with several 3D model style-compatibility results, and with an interactive system for modeling style-consistent scenes.
  arXiv  Detail & Related papers  (2020-10-20T18:19:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.