Generative AI for Urban Design: A Stepwise Approach Integrating Human Expertise with Multimodal Diffusion Models

• URL: http://arxiv.org/abs/2505.24260v1
• Date: Fri, 30 May 2025 06:33:48 GMT
• Title: Generative AI for Urban Design: A Stepwise Approach Integrating Human Expertise with Multimodal Diffusion Models
• Authors: Mingyi He, Yuebing Liang, Shenhao Wang, Yunhan Zheng, Qingyi Wang, Dingyi Zhuang, Li Tian, Jinhua Zhao,
• Abstract summary: This study proposes a stepwise generative urban design framework that integrates multimodal diffusion models with human expertise.<n>Rather than generating outcomes in a single end-to-end process, the framework divides the process into three key stages aligned with established urban design.<n>At each stage, rendering diffusion models generate preliminary designs based on textual prompts and image-based constraints.
• Score: 16.15278208238539
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Urban design is a multifaceted process that demands careful consideration of site-specific constraints and collaboration among diverse professionals and stakeholders. The advent of generative artificial intelligence (GenAI) offers transformative potential by improving the efficiency of design generation and facilitating the communication of design ideas. However, most existing approaches are not well integrated with human design workflows. They often follow end-to-end pipelines with limited control, overlooking the iterative nature of real-world design. This study proposes a stepwise generative urban design framework that integrates multimodal diffusion models with human expertise to enable more adaptive and controllable design processes. Instead of generating design outcomes in a single end-to-end process, the framework divides the process into three key stages aligned with established urban design workflows: (1) road network and land use planning, (2) building layout planning, and (3) detailed planning and rendering. At each stage, multimodal diffusion models generate preliminary designs based on textual prompts and image-based constraints, which can then be reviewed and refined by human designers. We design an evaluation framework to assess the fidelity, compliance, and diversity of the generated designs. Experiments using data from Chicago and New York City demonstrate that our framework outperforms baseline models and end-to-end approaches across all three dimensions. This study underscores the benefits of multimodal diffusion models and stepwise generation in preserving human control and facilitating iterative refinements, laying the groundwork for human-AI interaction in urban design solutions.

Related papers

• CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Design [69.83433430133302]
  CreatiDesign is a systematic solution for automated graphic design covering both model architecture and dataset construction.<n>First, we design a unified multi-condition driven architecture that enables flexible and precise integration of heterogeneous design elements.<n> Furthermore, to ensure that each condition precisely controls its designated image region, we propose a multimodal attention mask mechanism.
  arXiv  Detail & Related papers  (2025-05-25T12:14:23Z)
• I-Design: Personalized LLM Interior Designer [57.00412237555167]
  I-Design is a personalized interior designer that allows users to generate and visualize their design goals through natural language communication. I-Design starts with a team of large language model agents that engage in dialogues and logical reasoning with one another. The final design is then constructed in 3D by retrieving and integrating assets from an existing object database.
  arXiv  Detail & Related papers  (2024-04-03T16:17:53Z)
• Diffusion Model for Data-Driven Black-Box Optimization [54.25693582870226]
  We focus on diffusion models, a powerful generative AI technology, and investigate their potential for black-box optimization. We study two practical types of labels: 1) noisy measurements of a real-valued reward function and 2) human preference based on pairwise comparisons. Our proposed method reformulates the design optimization problem into a conditional sampling problem, which allows us to leverage the power of diffusion models.
  arXiv  Detail & Related papers  (2024-03-20T00:41:12Z)
• Prototyping with Prompts: Emerging Approaches and Challenges in Generative AI Design for Collaborative Software Teams [2.237039275844699]
  Generative AI models are increasingly being integrated into human task, enabling the production of expressive content.<n>Unlike traditional human-AI design methods, the new approach to designing generative capabilities focuses heavily on prompt engineering strategies.<n>Our findings highlight emerging practices and role shifts in AI system prototyping among multistakeholder teams.
  arXiv  Detail & Related papers  (2024-02-27T17:56:10Z)
• Compositional Generative Inverse Design [69.22782875567547]
  Inverse design, where we seek to design input variables in order to optimize an underlying objective function, is an important problem. We show that by instead optimizing over the learned energy function captured by the diffusion model, we can avoid such adversarial examples. In an N-body interaction task and a challenging 2D multi-airfoil design task, we demonstrate that by composing the learned diffusion model at test time, our method allows us to design initial states and boundary shapes.
  arXiv  Detail & Related papers  (2024-01-24T01:33:39Z)
• DesignGPT: Multi-Agent Collaboration in Design [4.6272626111555955]
  DesignGPT uses artificial intelligence agents to simulate the roles of different positions in the design company and allows human designers to collaborate with them in natural language. Experimental results show that compared with separate AI tools, DesignGPT improves the performance of designers.
  arXiv  Detail & Related papers  (2023-11-20T08:05:52Z)
• Generative methods for Urban design and rapid solution space exploration [13.222198221605701]
  This research introduces an implementation of a tensor-field-based generative urban modeling toolkit. Our method encodes contextual constraints such as waterfront edges, terrain, view-axis, existing streets, landmarks, and non-geometric design inputs. This allows users to generate many, diverse urban fabric configurations that resemble real-world cities with very few model inputs.
  arXiv  Detail & Related papers  (2022-12-13T17:58:02Z)
• Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques [55.492211642128446]
  Designers reportedly struggle with design optimization tasks where they are asked to find a combination of design parameters that maximizes a given set of objectives. Model-based computational design algorithms assist designers by generating design examples during design. Black box methods for assistance, on the other hand, can work with any design problem.
  arXiv  Detail & Related papers  (2022-04-15T20:40:43Z)
• iPLAN: Interactive and Procedural Layout Planning [13.172253981084403]
  We propose a new human-in-the-loop generative model, iPLAN. It is capable of automatically generating layouts, but also interacting with designers throughout the whole procedure. The results show that iPLAN has high fidelity in producing similar layouts to those from human designers.
  arXiv  Detail & Related papers  (2022-03-27T23:21:15Z)
• Dynamically Grown Generative Adversarial Networks [111.43128389995341]
  We propose a method to dynamically grow a GAN during training, optimizing the network architecture and its parameters together with automation. The method embeds architecture search techniques as an interleaving step with gradient-based training to periodically seek the optimal architecture-growing strategy for the generator and discriminator.
  arXiv  Detail & Related papers  (2021-06-16T01:25:51Z)
• MO-PaDGAN: Generating Diverse Designs with Multivariate Performance Enhancement [13.866787416457454]
  Deep generative models have proven useful for automatic design synthesis and design space exploration. They face three challenges when applied to engineering design: 1) generated designs lack diversity, 2) it is difficult to explicitly improve all the performance measures of generated designs, and 3) existing models generally do not generate high-performance novel designs. We propose MO-PaDGAN, which contains a new Determinantal Point Processes based loss function for probabilistic modeling of diversity and performances.
  arXiv  Detail & Related papers  (2020-07-07T21:57:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.