LLM4DESIGN: An Automated Multi-Modal System for Architectural and Environmental Design

• URL: http://arxiv.org/abs/2407.12025v1
• Date: Fri, 28 Jun 2024 10:57:50 GMT
• Title: LLM4DESIGN: An Automated Multi-Modal System for Architectural and Environmental Design
• Authors: Ran Chen, Xueqi Yao, Xuhui Jiang,
• Abstract summary: This study introduces LLM4DESIGN, a highly automated system for generating architectural and environmental design proposals. It employs Multi-Agent systems to foster creativity, Retrieval Augmented Generation (RAG) to ground designs in realism, and Visual Language Models (VLM) to synchronize all information.
• Score: 2.934816617441014
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This study introduces LLM4DESIGN, a highly automated system for generating architectural and environmental design proposals. LLM4DESIGN, relying solely on site conditions and design requirements, employs Multi-Agent systems to foster creativity, Retrieval Augmented Generation (RAG) to ground designs in realism, and Visual Language Models (VLM) to synchronize all information. This system resulting in coherent, multi-illustrated, and multi-textual design schemes. The system meets the dual needs of narrative storytelling and objective drawing presentation in generating architectural and environmental design proposals. Extensive comparative and ablation experiments confirm the innovativeness of LLM4DESIGN's narrative and the grounded applicability of its plans, demonstrating its superior performance in the field of urban renewal design. Lastly, we have created the first cross-modal design scheme dataset covering architecture, landscape, interior, and urban design, providing rich resources for future research.

Related papers

• Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs [56.391404083287235]
  We introduce Cambrian-1, a family of multimodal LLMs (MLLMs) designed with a vision-centric approach. Our study uses LLMs and visual instruction tuning as an interface to evaluate various visual representations. We provide model weights, code, supporting tools, datasets, and detailed instruction-tuning and evaluation recipes.
  arXiv  Detail & Related papers  (2024-06-24T17:59:42Z)
• LLM2FEA: Discover Novel Designs with Generative Evolutionary Multitasking [21.237950330178354]
  We propose the first attempt to discover novel designs in generative models by transferring knowledge across multiple domains. By utilizing a multi-factorial evolutionary algorithm (MFEA) to drive a large language model, LLM2FEA integrates knowledge from various fields to generate prompts that guide the generative model in discovering novel and practical objects.
  arXiv  Detail & Related papers  (2024-06-21T07:20:51Z)
• PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with LLM [58.67882997399021]
  Our research introduces a unified framework for automated graphic layout generation. Our data-driven method employs structured text (JSON format) and visual instruction tuning to generate layouts. We conduct extensive experiments and achieved state-of-the-art (SOTA) performance on public multi-modal layout generation benchmarks.
  arXiv  Detail & Related papers  (2024-06-05T03:05:52Z)
• Generating Daylight-driven Architectural Design via Diffusion Models [2.3020018305241337]
  We present a novel daylight-driven AI-aided architectural design method. Firstly, we formulate a method for generating massing models, producing architectural massing models using random parameters. We integrate a daylight-driven facade design strategy, accurately determining window layouts and applying them to the massing models.
  arXiv  Detail & Related papers  (2024-04-20T11:28:14Z)
• I-Design: Personalized LLM Interior Designer [57.00412237555167]
  I-Design is a personalized interior designer that allows users to generate and visualize their design goals through natural language communication. I-Design starts with a team of large language model agents that engage in dialogues and logical reasoning with one another. The final design is then constructed in 3D by retrieving and integrating assets from an existing object database.
  arXiv  Detail & Related papers  (2024-04-03T16:17:53Z)
• Zero-shot Sequential Neuro-symbolic Reasoning for Automatically Generating Architecture Schematic Designs [4.78070970632469]
  This paper introduces a novel automated system for generating architecture schematic designs. We leverage the combined strengths of generative AI (neuro reasoning) and mathematical program solvers (symbolic reasoning) Our method can generate various building designs in accordance with the understanding of the neighborhood.
  arXiv  Detail & Related papers  (2024-01-25T12:52:42Z)
• Compositional Foundation Models for Hierarchical Planning [52.18904315515153]
  We propose a foundation model which leverages expert foundation model trained on language, vision and action data individually together to solve long-horizon tasks. We use a large language model to construct symbolic plans that are grounded in the environment through a large video diffusion model. Generated video plans are then grounded to visual-motor control, through an inverse dynamics model that infers actions from generated videos.
  arXiv  Detail & Related papers  (2023-09-15T17:44:05Z)
• Experiments on Generative AI-Powered Parametric Modeling and BIM for Architectural Design [4.710049212041078]
  The study experiments with the potential of ChatGPT and generative AI in 3D architectural design. The framework provides architects with an intuitive and powerful method to convey design intent.
  arXiv  Detail & Related papers  (2023-08-01T01:51:59Z)
• Towards AI-Architecture Liberty: A Comprehensive Survey on Design and Generation of Virtual Architecture by Deep Learning [23.58793497403681]
  3D shape generation techniques leveraging deep learning have garnered significant interest from both the computer vision and architectural design communities. We review 149 related articles covering architectural design, 3D shape techniques, and virtual environments. We highlight four important enablers of ubiquitous interaction with immersive systems in deep learning-assisted architectural generation.
  arXiv  Detail & Related papers  (2023-04-30T15:38:36Z)
• Dynamically Grown Generative Adversarial Networks [111.43128389995341]
  We propose a method to dynamically grow a GAN during training, optimizing the network architecture and its parameters together with automation. The method embeds architecture search techniques as an interleaving step with gradient-based training to periodically seek the optimal architecture-growing strategy for the generator and discriminator.
  arXiv  Detail & Related papers  (2021-06-16T01:25:51Z)
• Twins: Revisiting Spatial Attention Design in Vision Transformers [81.02454258677714]
  In this work, we demonstrate that a carefully-devised yet simple spatial attention mechanism performs favourably against the state-of-the-art schemes. We propose two vision transformer architectures, namely, Twins-PCPVT and Twins-SVT. Our proposed architectures are highly-efficient and easy to implement, only involving matrix multiplications that are highly optimized in modern deep learning frameworks.
  arXiv  Detail & Related papers  (2021-04-28T15:42:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.