Step-by-step Layered Design Generation

• URL: http://arxiv.org/abs/2512.03335v1
• Date: Wed, 03 Dec 2025 00:59:43 GMT
• Title: Step-by-step Layered Design Generation
• Authors: Faizan Farooq Khan, K J Joseph, Koustava Goswami, Mohamed Elhoseiny, Balaji Vasan Srinivasan,
• Abstract summary: We propose a novel problem setting called Step-by-Step Layered Design Generation.<n>It tasks a machine learning model with generating a design that adheres to a sequence of instructions from a designer.<n>To complement our new problem setting, we introduce a new evaluation suite, including a dataset and a benchmark.
• Score: 47.423344283764074
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Design generation, in its essence, is a step-by-step process where designers progressively refine and enhance their work through careful modifications. Despite this fundamental characteristic, existing approaches mainly treat design synthesis as a single-step generation problem, significantly underestimating the inherent complexity of the creative process. To bridge this gap, we propose a novel problem setting called Step-by-Step Layered Design Generation, which tasks a machine learning model with generating a design that adheres to a sequence of instructions from a designer. Leveraging recent advancements in multi-modal LLMs, we propose SLEDGE: Step-by-step LayEred Design GEnerator to model each update to a design as an atomic, layered change over its previous state, while being grounded in the instruction. To complement our new problem setting, we introduce a new evaluation suite, including a dataset and a benchmark. Our exhaustive experimental analysis and comparison with state-of-the-art approaches tailored to our new setup demonstrate the efficacy of our approach. We hope our work will attract attention to this pragmatic and under-explored research area.

Related papers

• PosterCopilot: Toward Layout Reasoning and Controllable Editing for Professional Graphic Design [29.264277919540955]
  PosterCopilot is a framework that advances layout reasoning and controllable editing for professional graphic design.<n>We introduce a progressive three-stage training strategy that equips LMMs with geometric understanding and aesthetic reasoning for layout design.<n>We develop a complete workflow that couples the trained LMM-based design model with generative models, enabling layer-controllable, iterative editing for precise element refinement.
  arXiv  Detail & Related papers  (2025-12-03T18:59:37Z)
• Computer-Aided Layout Generation for Building Design: A Review [18.703604111298695]
  In this paper, we conduct a review of three major research topics of architecture layout design and generation.<n>For each topic, we present an overview of the leading paradigms, categorized either by research domains (architecture or machine learning) or by user input conditions or constraints.
  arXiv  Detail & Related papers  (2025-04-13T19:00:53Z)
• From Elements to Design: A Layered Approach for Automatic Graphic Design Composition [16.262338090888342]
  We introduce the layered design principle into Large Multimodal Models (LMMs)<n>LaDeCo performs layer planning for a given element set, dividing the input elements into different semantic layers according to their contents.<n>It subsequently predicts element attributes that control the design composition in a layer-wise manner, and includes the rendered image of previously generated layers into the context.
  arXiv  Detail & Related papers  (2024-12-27T16:13:08Z)
• STAR: Synthesis of Tailored Architectures [61.080157488857516]
  We propose a new approach for the synthesis of tailored architectures (STAR)<n>Our approach combines a novel search space based on the theory of linear input-varying systems, supporting a hierarchical numerical encoding into architecture genomes. STAR genomes are automatically refined and recombined with gradient-free, evolutionary algorithms to optimize for multiple model quality and efficiency metrics.<n>Using STAR, we optimize large populations of new architectures, leveraging diverse computational units and interconnection patterns, improving over highly-optimized Transformers and striped hybrid models on the frontier of quality, parameter size, and inference cache for autoregressive language modeling.
  arXiv  Detail & Related papers  (2024-11-26T18:42:42Z)
• Compositional Generative Inverse Design [69.22782875567547]
  Inverse design, where we seek to design input variables in order to optimize an underlying objective function, is an important problem. We show that by instead optimizing over the learned energy function captured by the diffusion model, we can avoid such adversarial examples. In an N-body interaction task and a challenging 2D multi-airfoil design task, we demonstrate that by composing the learned diffusion model at test time, our method allows us to design initial states and boundary shapes.
  arXiv  Detail & Related papers  (2024-01-24T01:33:39Z)
• Representation Learning for Sequential Volumetric Design Tasks [11.702880690338677]
  We propose to encode the design knowledge from a collection of expert or high-performing design sequences.<n>We develop the preference model by estimating the density of the learned representations.<n>We train an autoregressive transformer model for sequential design generation.
  arXiv  Detail & Related papers  (2023-09-05T21:21:06Z)
• Heterogeneous Continual Learning [88.53038822561197]
  We propose a novel framework to tackle the continual learning (CL) problem with changing network architectures. We build on top of the distillation family of techniques and modify it to a new setting where a weaker model takes the role of a teacher. We also propose Quick Deep Inversion (QDI) to recover prior task visual features to support knowledge transfer.
  arXiv  Detail & Related papers  (2023-06-14T15:54:42Z)
• Design Space Exploration and Explanation via Conditional Variational Autoencoders in Meta-model-based Conceptual Design of Pedestrian Bridges [52.77024349608834]
  This paper provides a performance-driven design exploration framework to augment the human designer through a Conditional Variational Autoencoder (CVAE) The CVAE is trained on 18'000 synthetically generated instances of a pedestrian bridge in Switzerland.
  arXiv  Detail & Related papers  (2022-11-29T17:28:31Z)
• Dynamically Grown Generative Adversarial Networks [111.43128389995341]
  We propose a method to dynamically grow a GAN during training, optimizing the network architecture and its parameters together with automation. The method embeds architecture search techniques as an interleaving step with gradient-based training to periodically seek the optimal architecture-growing strategy for the generator and discriminator.
  arXiv  Detail & Related papers  (2021-06-16T01:25:51Z)
• Generative Design by Reinforcement Learning: Enhancing the Diversity of Topology Optimization Designs [5.8010446129208155]
  This study proposes a reinforcement learning based generative design process, with reward functions maximizing the diversity of topology designs. We show that RL-based generative design produces a large number of diverse designs within a short inference time by exploiting GPU in a fully automated manner.
  arXiv  Detail & Related papers  (2020-08-17T06:50:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.