Text2CAD: Generating Sequential CAD Models from Beginner-to-Expert Level Text Prompts

• URL: http://arxiv.org/abs/2409.17106v1
• Date: Wed, 25 Sep 2024 17:19:33 GMT
• Title: Text2CAD: Generating Sequential CAD Models from Beginner-to-Expert Level Text Prompts
• Authors: Mohammad Sadil Khan, Sankalp Sinha, Talha Uddin Sheikh, Didier Stricker, Sk Aziz Ali, Muhammad Zeshan Afzal,
• Abstract summary: We propose Text2CAD, the first AI framework for generating text-to-parametric CAD models. Our proposed framework shows great potential in AI-aided design applications.
• Score: 12.63158811936688
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Prototyping complex computer-aided design (CAD) models in modern softwares can be very time-consuming. This is due to the lack of intelligent systems that can quickly generate simpler intermediate parts. We propose Text2CAD, the first AI framework for generating text-to-parametric CAD models using designer-friendly instructions for all skill levels. Furthermore, we introduce a data annotation pipeline for generating text prompts based on natural language instructions for the DeepCAD dataset using Mistral and LLaVA-NeXT. The dataset contains $\sim170$K models and $\sim660$K text annotations, from abstract CAD descriptions (e.g., generate two concentric cylinders) to detailed specifications (e.g., draw two circles with center $(x,y)$ and radius $r_{1}$, $r_{2}$, and extrude along the normal by $d$...). Within the Text2CAD framework, we propose an end-to-end transformer-based auto-regressive network to generate parametric CAD models from input texts. We evaluate the performance of our model through a mixture of metrics, including visual quality, parametric precision, and geometrical accuracy. Our proposed framework shows great potential in AI-aided design applications. Our source code and annotations will be publicly available.

Related papers

• Text2CAD: Text to 3D CAD Generation via Technical Drawings [45.3611544056261]
  Text2CAD is a novel framework that employs stable diffusion models tailored to automate the generation process. We show that Text2CAD effectively generates technical drawings that are accurately translated into high-quality 3D CAD models.
  arXiv  Detail & Related papers  (2024-11-09T15:12:06Z)
• CAD-MLLM: Unifying Multimodality-Conditioned CAD Generation With MLLM [39.113795259823476]
  We introduce the CAD-MLLM, the first system capable of generating parametric CAD models conditioned on the multimodal input. We use advanced large language models (LLMs) to align the feature space across diverse multi-modalities data and CAD models' vectorized representations. Our resulting dataset, named Omni-CAD, is the first multimodal CAD dataset that contains textual description, multi-view images, points, and command sequence for each CAD model.
  arXiv  Detail & Related papers  (2024-11-07T18:31:08Z)
• FlexCAD: Unified and Versatile Controllable CAD Generation with Fine-tuned Large Language Models [22.010338370150738]
  There is a growing interest in creating computer-aided design (CAD) models based on user intent. Existing work offers limited controllability and needs separate models for different types of control. We propose FlexCAD, a unified model by fine-tuning large language models.
  arXiv  Detail & Related papers  (2024-11-05T05:45:26Z)
• Img2CAD: Conditioned 3D CAD Model Generation from Single Image with Structured Visual Geometry [12.265852643914439]
  We present Img2CAD, the first knowledge that uses 2D image inputs to generate editable parameters. Img2CAD enables seamless integration between AI 3D reconstruction and CAD representation.
  arXiv  Detail & Related papers  (2024-10-04T13:27:52Z)
• PS-CAD: Local Geometry Guidance via Prompting and Selection for CAD Reconstruction [86.726941702182]
  We introduce geometric guidance into the reconstruction network PS-CAD. We provide the geometry of surfaces where the current reconstruction differs from the complete model as a point cloud. Second, we use geometric analysis to extract a set of planar prompts, that correspond to candidate surfaces.
  arXiv  Detail & Related papers  (2024-05-24T03:43:55Z)
• AWOL: Analysis WithOut synthesis using Language [57.31874938870305]
  We leverage language to control existing 3D shape models to produce novel shapes. We show that we can use text to generate new animals not present during training. This work also constitutes the first language-driven method for generating 3D trees.
  arXiv  Detail & Related papers  (2024-04-03T20:04:44Z)
• Model2Scene: Learning 3D Scene Representation via Contrastive Language-CAD Models Pre-training [105.3421541518582]
  Current successful methods of 3D scene perception rely on the large-scale annotated point cloud. We propose Model2Scene, a novel paradigm that learns free 3D scene representation from Computer-Aided Design (CAD) models and languages. Model2Scene yields impressive label-free 3D object salient detection with an average mAP of 46.08% and 55.49% on the ScanNet and S3DIS datasets, respectively.
  arXiv  Detail & Related papers  (2023-09-29T03:51:26Z)
• SECAD-Net: Self-Supervised CAD Reconstruction by Learning Sketch-Extrude Operations [21.000539206470897]
  SECAD-Net is an end-to-end neural network aimed at reconstructing compact and easy-to-edit CAD models. We show superiority over state-of-the-art alternatives including the closely related method for supervised CAD reconstruction.
  arXiv  Detail & Related papers  (2023-03-19T09:26:03Z)
• AutoCAD: Automatically Generating Counterfactuals for Mitigating Shortcut Learning [70.70393006697383]
  We present AutoCAD, a fully automatic and task-agnostic CAD generation framework. In this paper, we present AutoCAD, a fully automatic and task-agnostic CAD generation framework.
  arXiv  Detail & Related papers  (2022-11-29T13:39:53Z)
• GENIUS: Sketch-based Language Model Pre-training via Extreme and Selective Masking for Text Generation and Augmentation [76.7772833556714]
  We introduce GENIUS: a conditional text generation model using sketches as input. GENIUS is pre-trained on a large-scale textual corpus with a novel reconstruction from sketch objective. We show that GENIUS can be used as a strong and ready-to-use data augmentation tool for various natural language processing (NLP) tasks.
  arXiv  Detail & Related papers  (2022-11-18T16:39:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.