Intelligent Home 3D: Automatic 3D-House Design from Linguistic Descriptions Only

• URL: http://arxiv.org/abs/2003.00397v1
• Date: Sun, 1 Mar 2020 04:28:48 GMT
• Title: Intelligent Home 3D: Automatic 3D-House Design from Linguistic Descriptions Only
• Authors: Qi Chen, Qi Wu, Rui Tang, Yuhan Wang, Shuai Wang, Mingkui Tan
• Abstract summary: We formulate it as a language conditioned visual content generation problem that is divided into a floor plan generation and an interior texture synthesis task. To train and evaluate our model, we build the first Text-to-3D House Model dataset.
• Score: 55.3363844662966
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Home design is a complex task that normally requires architects to finish with their professional skills and tools. It will be fascinating that if one can produce a house plan intuitively without knowing much knowledge about home design and experience of using complex designing tools, for example, via natural language. In this paper, we formulate it as a language conditioned visual content generation problem that is further divided into a floor plan generation and an interior texture (such as floor and wall) synthesis task. The only control signal of the generation process is the linguistic expression given by users that describe the house details. To this end, we propose a House Plan Generative Model (HPGM) that first translates the language input to a structural graph representation and then predicts the layout of rooms with a Graph Conditioned Layout Prediction Network (GC LPN) and generates the interior texture with a Language Conditioned Texture GAN (LCT-GAN). With some post-processing, the final product of this task is a 3D house model. To train and evaluate our model, we build the first Text-to-3D House Model dataset.

Related papers

• SceneTeller: Language-to-3D Scene Generation [15.209079637302905]
  Given a prompt in natural language describing the object placement in the room, our method produces a high-quality 3D scene corresponding to it. Our turnkey pipeline produces state-of-the-art 3D scenes, while being easy to use even for novices.
  arXiv  Detail & Related papers  (2024-07-30T10:45:28Z)
• I-Design: Personalized LLM Interior Designer [57.00412237555167]
  I-Design is a personalized interior designer that allows users to generate and visualize their design goals through natural language communication. I-Design starts with a team of large language model agents that engage in dialogues and logical reasoning with one another. The final design is then constructed in 3D by retrieving and integrating assets from an existing object database.
  arXiv  Detail & Related papers  (2024-04-03T16:17:53Z)
• L3GO: Language Agents with Chain-of-3D-Thoughts for Generating Unconventional Objects [53.4874127399702]
  We propose a language agent with chain-of-3D-thoughts (L3GO), an inference-time approach that can reason about part-based 3D mesh generation. We develop a new benchmark, Unconventionally Feasible Objects (UFO), as well as SimpleBlenv, a wrapper environment built on top of Blender. Our approach surpasses the standard GPT-4 and other language agents for 3D mesh generation on ShapeNet.
  arXiv  Detail & Related papers  (2024-02-14T09:51:05Z)
• AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homes [10.482201110770584]
  We introduce AnyHome, a framework that translates any text into well-structured and textured indoor scenes at a house-scale. By prompting Large Language Models (LLMs) with designed templates, our approach converts provided textual narratives into amodal structured representations. A Score Distillation Sampling process is then employed to refine the geometry, followed by an egocentric inpainting process that adds lifelike textures to it.
  arXiv  Detail & Related papers  (2023-12-11T18:56:37Z)
• GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphs [74.98581417902201]
  We propose a novel framework to generate compositional 3D scenes from scene graphs. By exploiting node and edge information in scene graphs, our method makes better use of the pretrained text-to-image diffusion model. We conduct both qualitative and quantitative experiments to validate the effectiveness of GraphDreamer.
  arXiv  Detail & Related papers  (2023-11-30T18:59:58Z)
• Tell2Design: A Dataset for Language-Guided Floor Plan Generation [21.686370988228614]
  We consider the task of generating designs directly from natural language descriptions. Designs must satisfy different constraints that are not present in generating artistic images.
  arXiv  Detail & Related papers  (2023-11-27T15:49:29Z)
• 3D-GPT: Procedural 3D Modeling with Large Language Models [47.72968643115063]
  We introduce 3D-GPT, a framework utilizing large language models(LLMs) for instruction-driven 3D modeling. 3D-GPT positions LLMs as proficient problem solvers, dissecting the procedural 3D modeling tasks into accessible segments and appointing the apt agent for each task. Our empirical investigations confirm that 3D-GPT not only interprets and executes instructions, delivering reliable results but also collaborates effectively with human designers.
  arXiv  Detail & Related papers  (2023-10-19T17:41:48Z)
• Zero-Shot Text-Guided Object Generation with Dream Fields [111.06026544180398]
  We combine neural rendering with multi-modal image and text representations to synthesize diverse 3D objects. Our method, Dream Fields, can generate the geometry and color of a wide range of objects without 3D supervision. In experiments, Dream Fields produce realistic, multi-view consistent object geometry and color from a variety of natural language captions.
  arXiv  Detail & Related papers  (2021-12-02T17:53:55Z)
• LanguageRefer: Spatial-Language Model for 3D Visual Grounding [72.7618059299306]
  We develop a spatial-language model for a 3D visual grounding problem. We show that our model performs competitively on visio-linguistic datasets proposed by ReferIt3D.
  arXiv  Detail & Related papers  (2021-07-07T18:55:03Z)
• Building-GAN: Graph-Conditioned Architectural Volumetric Design Generation [10.024367148266721]
  This paper focuses on volumetric design generation conditioned on an input program graph. Instead of outputting dense 3D voxels, we propose a new 3D representation named voxel graph that is both compact and expressive for building geometries. Our generator is a cross-modal graph neural network that uses a pointer mechanism to connect the input program graph and the output voxel graph, and the whole pipeline is trained using the adversarial framework.
  arXiv  Detail & Related papers  (2021-04-27T16:49:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.