Generative AI-based Prompt Evolution Engineering Design Optimization With Vision-Language Model

• URL: http://arxiv.org/abs/2406.09143v2
• Date: Fri, 14 Jun 2024 08:33:11 GMT
• Title: Generative AI-based Prompt Evolution Engineering Design Optimization With Vision-Language Model
• Authors: Melvin Wong, Thiago Rios, Stefan Menzel, Yew Soon Ong,
• Abstract summary: We present a prompt evolution design optimization (PEDO) framework contextualized in a vehicle design scenario. We use a physics-based solver and a vision-language model for practical or functional guidance in the generated car designs. Our investigations on a car design optimization problem show a wide spread of potential car designs generated at the early phase of the search.
• Score: 22.535058343006828
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Engineering design optimization requires an efficient combination of a 3D shape representation, an optimization algorithm, and a design performance evaluation method, which is often computationally expensive. We present a prompt evolution design optimization (PEDO) framework contextualized in a vehicle design scenario that leverages a vision-language model for penalizing impractical car designs synthesized by a generative model. The backbone of our framework is an evolutionary strategy coupled with an optimization objective function that comprises a physics-based solver and a vision-language model for practical or functional guidance in the generated car designs. In the prompt evolutionary search, the optimizer iteratively generates a population of text prompts, which embed user specifications on the aerodynamic performance and visual preferences of the 3D car designs. Then, in addition to the computational fluid dynamics simulations, the pre-trained vision-language model is used to penalize impractical designs and, thus, foster the evolutionary algorithm to seek more viable designs. Our investigations on a car design optimization problem show a wide spread of potential car designs generated at the early phase of the search, which indicates a good diversity of designs in the initial populations, and an increase of over 20\% in the probability of generating practical designs compared to a baseline framework without using a vision-language model. Visual inspection of the designs against the performance results demonstrates prompt evolution as a very promising paradigm for finding novel designs with good optimization performance while providing ease of use in specifying design specifications and preferences via a natural language interface.

Related papers

• Cliqueformer: Model-Based Optimization with Structured Transformers [102.55764949282906]
  We develop a model that learns the structure of an MBO task and empirically leads to improved designs. We evaluate Cliqueformer on various tasks, ranging from high-dimensional black-box functions to real-world tasks of chemical and genetic design.
  arXiv  Detail & Related papers  (2024-10-17T00:35:47Z)
• Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models [54.132297393662654]
  We introduce a hybrid method that fine-tunes cutting-edge diffusion models by optimizing reward models through RL. We demonstrate the capability of our approach to outperform the best designs in offline data, leveraging the extrapolation capabilities of reward models.
  arXiv  Detail & Related papers  (2024-05-30T03:57:29Z)
• Diffusion Model for Data-Driven Black-Box Optimization [54.25693582870226]
  We focus on diffusion models, a powerful generative AI technology, and investigate their potential for black-box optimization. We study two practical types of labels: 1) noisy measurements of a real-valued reward function and 2) human preference based on pairwise comparisons. Our proposed method reformulates the design optimization problem into a conditional sampling problem, which allows us to leverage the power of diffusion models.
  arXiv  Detail & Related papers  (2024-03-20T00:41:12Z)
• Compositional Generative Inverse Design [69.22782875567547]
  Inverse design, where we seek to design input variables in order to optimize an underlying objective function, is an important problem. We show that by instead optimizing over the learned energy function captured by the diffusion model, we can avoid such adversarial examples. In an N-body interaction task and a challenging 2D multi-airfoil design task, we demonstrate that by composing the learned diffusion model at test time, our method allows us to design initial states and boundary shapes.
  arXiv  Detail & Related papers  (2024-01-24T01:33:39Z)
• Diffusion Generative Inverse Design [28.04683283070957]
  Inverse design refers to the problem of optimizing the input of an objective function in order to enact a target outcome. Recent developments in learned graph neural networks (GNNs) can be used for accurate, efficient, differentiable estimation of simulator dynamics. We show how denoising diffusion diffusion models can be used to solve inverse design problems efficiently and propose a particle sampling algorithm for further improving their efficiency.
  arXiv  Detail & Related papers  (2023-09-05T08:32:07Z)
• Large Language and Text-to-3D Models for Engineering Design Optimization [0.1740313383876245]
  We study the potential of deep text-to-3D models in the engineering domain. We use Shap-E, a text-to-3D asset network by OpenAI, in the context of aerodynamic vehicle optimization.
  arXiv  Detail & Related papers  (2023-07-03T07:54:09Z)
• Aligning Optimization Trajectories with Diffusion Models for Constrained Design Generation [17.164961143132473]
  We introduce a learning framework that demonstrates the efficacy of aligning the sampling trajectory of diffusion models with the optimization trajectory derived from traditional physics-based methods. Our method allows for generating feasible and high-performance designs in as few as two steps without the need for expensive preprocessing, external surrogate models, or additional labeled data. Our results demonstrate that TA outperforms state-of-the-art deep generative models on in-distribution configurations and halves the inference computational cost.
  arXiv  Detail & Related papers  (2023-05-29T09:16:07Z)
• Design Space Exploration and Explanation via Conditional Variational Autoencoders in Meta-model-based Conceptual Design of Pedestrian Bridges [52.77024349608834]
  This paper provides a performance-driven design exploration framework to augment the human designer through a Conditional Variational Autoencoder (CVAE) The CVAE is trained on 18'000 synthetically generated instances of a pedestrian bridge in Switzerland.
  arXiv  Detail & Related papers  (2022-11-29T17:28:31Z)
• Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques [55.492211642128446]
  Designers reportedly struggle with design optimization tasks where they are asked to find a combination of design parameters that maximizes a given set of objectives. Model-based computational design algorithms assist designers by generating design examples during design. Black box methods for assistance, on the other hand, can work with any design problem.
  arXiv  Detail & Related papers  (2022-04-15T20:40:43Z)
• Early-Phase Performance-Driven Design using Generative Models [0.0]
  This research introduces a novel method for performance-driven geometry generation that can afford interaction directly in the 3d modeling environment. The method uses Machine Learning techniques to train a generative model offline. By navigating the generative model's latent space, geometries with the desired characteristics can be quickly generated.
  arXiv  Detail & Related papers  (2021-07-19T01:25:11Z)
• Generative Design by Reinforcement Learning: Enhancing the Diversity of Topology Optimization Designs [5.8010446129208155]
  This study proposes a reinforcement learning based generative design process, with reward functions maximizing the diversity of topology designs. We show that RL-based generative design produces a large number of diverse designs within a short inference time by exploiting GPU in a fully automated manner.
  arXiv  Detail & Related papers  (2020-08-17T06:50:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.