How Can Large Language Models Help Humans in Design and Manufacturing?

• URL: http://arxiv.org/abs/2307.14377v1
• Date: Tue, 25 Jul 2023 17:30:38 GMT
• Title: How Can Large Language Models Help Humans in Design and Manufacturing?
• Authors: Liane Makatura, Michael Foshey, Bohan Wang, Felix H\"ahnLein, Pingchuan Ma, Bolei Deng, Megan Tjandrasuwita, Andrew Spielberg, Crystal Elaine Owens, Peter Yichen Chen, Allan Zhao, Amy Zhu, Wil J Norton, Edward Gu, Joshua Jacob, Yifei Li, Adriana Schulz, Wojciech Matusik
• Abstract summary: Large Language Models (LLMs), including GPT-4, provide exciting new opportunities for generative design. We scrutinize the utility of LLMs in tasks such as: converting a text-based prompt into a design specification, transforming a design into manufacturing instructions, producing a design space and design variations, computing the performance of a design, and searching for designs predicated on performance. By exposing these limitations, we aspire to catalyze the continued improvement and progression of these models.
• Score: 28.28959612862582
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The advancement of Large Language Models (LLMs), including GPT-4, provides exciting new opportunities for generative design. We investigate the application of this tool across the entire design and manufacturing workflow. Specifically, we scrutinize the utility of LLMs in tasks such as: converting a text-based prompt into a design specification, transforming a design into manufacturing instructions, producing a design space and design variations, computing the performance of a design, and searching for designs predicated on performance. Through a series of examples, we highlight both the benefits and the limitations of the current LLMs. By exposing these limitations, we aspire to catalyze the continued improvement and progression of these models.

Related papers

• Optimizing Token Usage on Large Language Model Conversations Using the Design Structure Matrix [49.1574468325115]
  Large Language Models become ubiquitous in many sectors and tasks. There is a need to reduce token usage, overcoming challenges such as short context windows, limited output sizes, and costs associated with token intake and generation. This work brings the Design Structure Matrix from the engineering design discipline into LLM conversation optimization.
  arXiv  Detail & Related papers  (2024-10-01T14:38:36Z)
• PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with LLM [58.67882997399021]
  Our research introduces a unified framework for automated graphic layout generation. Our data-driven method employs structured text (JSON format) and visual instruction tuning to generate layouts. We conduct extensive experiments and achieved state-of-the-art (SOTA) performance on public multi-modal layout generation benchmarks.
  arXiv  Detail & Related papers  (2024-06-05T03:05:52Z)
• Automatic Layout Planning for Visually-Rich Documents with Instruction-Following Models [81.6240188672294]
  In graphic design, non-professional users often struggle to create visually appealing layouts due to limited skills and resources. We introduce a novel multimodal instruction-following framework for layout planning, allowing users to easily arrange visual elements into tailored layouts. Our method not only simplifies the design process for non-professionals but also surpasses the performance of few-shot GPT-4V models, with mIoU higher by 12% on Crello.
  arXiv  Detail & Related papers  (2024-04-23T17:58:33Z)
• DesignProbe: A Graphic Design Benchmark for Multimodal Large Language Models [35.10231741092462]
  A well-executed graphic design typically achieves harmony in two levels, from the fine-grained design elements (color, font and layout) to the overall design. With the rapid development of Multimodal Large Language Models (MLLMs), we establish the DesignProbe, a benchmark to investigate the capability of MLLMs in design.
  arXiv  Detail & Related papers  (2024-04-23T07:31:19Z)
• I-Design: Personalized LLM Interior Designer [57.00412237555167]
  I-Design is a personalized interior designer that allows users to generate and visualize their design goals through natural language communication. I-Design starts with a team of large language model agents that engage in dialogues and logical reasoning with one another. The final design is then constructed in 3D by retrieving and integrating assets from an existing object database.
  arXiv  Detail & Related papers  (2024-04-03T16:17:53Z)
• Geometric Deep Learning for Computer-Aided Design: A Survey [85.79012726689511]
  This survey offers a comprehensive overview of learning-based methods in computer-aided design. It includes similarity analysis and retrieval, 2D and 3D CAD model synthesis, and CAD generation from point clouds. It provides a complete list of benchmark datasets and their characteristics, along with open-source codes that have propelled research in this domain.
  arXiv  Detail & Related papers  (2024-02-27T17:11:35Z)
• Compositional Generative Inverse Design [69.22782875567547]
  Inverse design, where we seek to design input variables in order to optimize an underlying objective function, is an important problem. We show that by instead optimizing over the learned energy function captured by the diffusion model, we can avoid such adversarial examples. In an N-body interaction task and a challenging 2D multi-airfoil design task, we demonstrate that by composing the learned diffusion model at test time, our method allows us to design initial states and boundary shapes.
  arXiv  Detail & Related papers  (2024-01-24T01:33:39Z)
• Zero-Shot RTL Code Generation with Attention Sink Augmented Large Language Models [0.0]
  This paper discusses the possibility of exploiting large language models to streamline the code generation process in hardware design. The ability to use large language models on RTL code generation not only expedites design cycles but also facilitates the exploration of design spaces.
  arXiv  Detail & Related papers  (2024-01-12T17:41:38Z)
• From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Design [5.268919870502001]
  This paper presents a comprehensive evaluation of vision-language models (VLMs) across a spectrum of engineering design tasks. Specifically in this paper, we assess the capabilities of two VLMs, GPT-4V and LLaVA 1.6 34B, in design tasks such as sketch similarity analysis, CAD generation, topology optimization, manufacturability assessment, and engineering textbook problems.
  arXiv  Detail & Related papers  (2023-11-21T15:20:48Z)
• Representation Learning for Sequential Volumetric Design Tasks [11.702880690338677]
  We propose to encode the design knowledge from a collection of expert or high-performing design sequences. We develop the preference model by estimating the density of the learned representations. We train an autoregressive transformer model for sequential design generation.
  arXiv  Detail & Related papers  (2023-09-05T21:21:06Z)
• Deep Generative Models in Engineering Design: A Review [1.933681537640272]
  We present a review and analysis of Deep Generative Learning models in engineering design. Recent DGMs have shown promising results in design applications like structural optimization, materials design, and shape synthesis.
  arXiv  Detail & Related papers  (2021-10-21T02:50:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.