From Idea to CAD: A Language Model-Driven Multi-Agent System for Collaborative Design

• URL: http://arxiv.org/abs/2503.04417v1
• Date: Thu, 06 Mar 2025 13:21:27 GMT
• Title: From Idea to CAD: A Language Model-Driven Multi-Agent System for Collaborative Design
• Authors: Felix Ocker, Stefan Menzel, Ahmed Sadik, Thiago Rios,
• Abstract summary: We present an approach that mirrors this team structure with a Vision Language Model (VLM)-based Multi Agent System.<n>A model is generated automatically from sketches and/ or textual descriptions.<n>The resulting model can be refined collaboratively in an iterative validation loop with the user.
• Score: 0.06749750044497731
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Creating digital models using Computer Aided Design (CAD) is a process that requires in-depth expertise. In industrial product development, this process typically involves entire teams of engineers, spanning requirements engineering, CAD itself, and quality assurance. We present an approach that mirrors this team structure with a Vision Language Model (VLM)-based Multi Agent System, with access to parametric CAD tooling and tool documentation. Combining agents for requirements engineering, CAD engineering, and vision-based quality assurance, a model is generated automatically from sketches and/ or textual descriptions. The resulting model can be refined collaboratively in an iterative validation loop with the user. Our approach has the potential to increase the effectiveness of design processes, both for industry experts and for hobbyists who create models for 3D printing. We demonstrate the potential of the architecture at the example of various design tasks and provide several ablations that show the benefits of the architecture's individual components.

Related papers

• CADCrafter: Generating Computer-Aided Design Models from Unconstrained Images [69.7768227804928]
  CADCrafter is an image-to-parametric CAD model generation framework that trains solely on synthetic textureless CAD data. We introduce a geometry encoder to accurately capture diverse geometric features. Our approach can robustly handle real unconstrained CAD images, and even generalize to unseen general objects.
  arXiv  Detail & Related papers  (2025-04-07T06:01:35Z)
• PHT-CAD: Efficient CAD Parametric Primitive Analysis with Progressive Hierarchical Tuning [52.681829043446044]
  ParaCAD comprises over 10 million annotated drawings for training and 3,000 real-world industrial drawings with complex topological structures and physical constraints for test. PHT-CAD is a novel 2D PPA framework that harnesses the modality alignment and reasoning capabilities of Vision-Language Models.
  arXiv  Detail & Related papers  (2025-03-23T17:24:32Z)
• Evaluation of Architectural Synthesis Using Generative AI [49.1574468325115]
  This paper presents a comparative evaluation of two systems: GPT-4o and Claude 3.5, in the task of architectural 3D synthesis.<n>We conduct a case study on two buildings from Palladio's Four Books of Architecture (1965): Villa Rotonda and Palazzo Porto.<n>We assess the systems' abilities in (1) interpreting 2D and 3D representations of buildings from drawings, (2) encoding the buildings into a CAD software script, and (3) self-improving based on outputs.
  arXiv  Detail & Related papers  (2025-03-04T18:39:28Z)
• BlenderLLM: Training Large Language Models for Computer-Aided Design with Self-improvement [45.19076032719869]
  We present BlenderLLM, a framework for training Large Language Models (LLMs) in Computer-Aided Design (CAD)<n>Our results reveal that existing models demonstrate significant limitations in generating accurate CAD scripts.<n>Through minimal instruction-based fine-tuning and iterative self-improvement, BlenderLLM significantly surpasses these models in both functionality and accuracy of CAD script generation.
  arXiv  Detail & Related papers  (2024-12-16T14:34:02Z)
• Text2CAD: Text to 3D CAD Generation via Technical Drawings [45.3611544056261]
  Text2CAD is a novel framework that employs stable diffusion models tailored to automate the generation process. We show that Text2CAD effectively generates technical drawings that are accurately translated into high-quality 3D CAD models.
  arXiv  Detail & Related papers  (2024-11-09T15:12:06Z)
• CadVLM: Bridging Language and Vision in the Generation of Parametric CAD Sketches [24.239470848849418]
  Parametric Computer-Aided Design (CAD) is central to contemporary mechanical design. We propose CadVLM, an end-to-end vision language model for CAD generation.
  arXiv  Detail & Related papers  (2024-09-26T01:22:29Z)
• GenCAD: Image-Conditioned Computer-Aided Design Generation with Transformer-Based Contrastive Representation and Diffusion Priors [3.796768352477804]
  The creation of manufacturable and editable 3D shapes through Computer-Aided Design (CAD) remains a highly manual and time-consuming task. This paper introduces GenCAD, a generative model that employs autoregressive transformers with a contrastive learning framework and latent diffusion models to transform image inputs into parametric CAD command sequences.
  arXiv  Detail & Related papers  (2024-09-08T23:49:11Z)
• OpenECAD: An Efficient Visual Language Model for Editable 3D-CAD Design [1.481550828146527]
  We fine-tuned pre-trained models to create OpenECAD models (0.55B, 0.89B, 2.4B and 3.1B) OpenECAD models can process images of 3D designs as input and generate highly structured 2D sketches and 3D construction commands. These outputs can be directly used with existing CAD tools' APIs to generate project files.
  arXiv  Detail & Related papers  (2024-06-14T10:47:52Z)
• I-Design: Personalized LLM Interior Designer [57.00412237555167]
  I-Design is a personalized interior designer that allows users to generate and visualize their design goals through natural language communication. I-Design starts with a team of large language model agents that engage in dialogues and logical reasoning with one another. The final design is then constructed in 3D by retrieving and integrating assets from an existing object database.
  arXiv  Detail & Related papers  (2024-04-03T16:17:53Z)
• Geometric Deep Learning for Computer-Aided Design: A Survey [85.79012726689511]
  This survey offers a comprehensive overview of learning-based methods in computer-aided design. It includes similarity analysis and retrieval, 2D and 3D CAD model synthesis, and CAD generation from point clouds. It provides a complete list of benchmark datasets and their characteristics, along with open-source codes that have propelled research in this domain.
  arXiv  Detail & Related papers  (2024-02-27T17:11:35Z)
• Design Space Exploration and Explanation via Conditional Variational Autoencoders in Meta-model-based Conceptual Design of Pedestrian Bridges [52.77024349608834]
  This paper provides a performance-driven design exploration framework to augment the human designer through a Conditional Variational Autoencoder (CVAE) The CVAE is trained on 18'000 synthetically generated instances of a pedestrian bridge in Switzerland.
  arXiv  Detail & Related papers  (2022-11-29T17:28:31Z)
• Computer-Aided Design as Language [16.79054488229662]
  We propose a machine learning model capable of automatically generating highly structured 2D sketches. Our method is a combination of a general-purpose language modeling technique alongside an off-the-shelf data serialization protocol. We show that our approach has enough flexibility to accommodate the complexity of the domain and performs well for both unconditional synthesis and image-to-sketch translation.
  arXiv  Detail & Related papers  (2021-05-06T15:43:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.