GuideCAD: A Lightweight Multimodal Framework for 3D CAD Model Generation via Prefix Embedding

• URL: http://arxiv.org/abs/2606.07024v1
• Date: Fri, 05 Jun 2026 08:14:28 GMT
• Title: GuideCAD: A Lightweight Multimodal Framework for 3D CAD Model Generation via Prefix Embedding
• Abstract summary: We propose GuideCAD, which leverages semantically rich visual-textual representations to generate 3D CAD models.<n>GuideCAD uses a mapping network that converts image embeddings into prefix embeddings, enabling a pretrained large language model to integrate visual and textual information.<n>Our experimental results show that GuideCAD generates comparably high-quality 3D CAD models while using approximately four times fewer parameters and achieving twice the training efficiency compared to fine-tuning approaches.
• Score: 6.339931887475017
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Multi-modal approaches used for 3D CAD generation require substantial computational resources, necessitating efficient training. To address this, we propose GuideCAD, which leverages semantically rich visual-textual representations having only a small number of trainable parameters to generate 3D CAD models. Specifically, GuideCAD uses a mapping network that converts image embeddings into prefix embeddings, enabling a pretrained large language model to integrate visual and textual information. As a result, a transformer-based decoder predicts the construction sequence using the visual-textual embeddings in order to generate the 3D CAD model. For experimental evaluation, we construct a new dataset, referred to as GuideCAD, which consists of text-image pairs. Each pair includes a text prompt that represents a 3D CAD construction sequence and its corresponding 3D CAD image. Our experimental results show that GuideCAD generates comparably high-quality 3D CAD models while using approximately four times fewer parameters and achieving twice the training efficiency compared to fine-tuning approaches. We have released the source code and dataset for our method at: https://github.com/mskimS2/GuideCAD

Related papers

• SldprtNet: A Large-Scale Multimodal Dataset for CAD Generation in Language-Driven 3D Design [26.634272863620975]
  We introduce SldprtNet, a large-scale dataset comprising over 242,000 industrial parts.<n>The dataset provides 3D models in both.step and.sldprt formats to support diverse training and testing.<n>It features carefully selected real-world industrial parts, supporting tools for scalable dataset expansion.
  arXiv  Detail & Related papers  (2026-03-13T15:47:08Z)
• DreamCAD: Scaling Multi-modal CAD Generation using Differentiable Parametric Surfaces [67.45867367326105]
  We propose DreamCAD, a multi-modal generative framework that directly produces editable BReps from point-level supervision.<n>DreamCAD represents each BRep as a set of parametric patches and uses a differentiable tessellation method to generate meshes.<n>DreamCAD achieves state-of-the-art performance on ABC and averse benchmarks across text, image, and point modalities.
  arXiv  Detail & Related papers  (2026-03-05T19:04:07Z)
• PatchAlign3D: Local Feature Alignment for Dense 3D Shape understanding [67.15800065888887]
  Current foundation models for 3D shapes excel at global tasks (retrieval, classification) but transfer poorly to local part-level reasoning.<n>We introduce an encoder-only 3D model that produces language-aligned patch-level features directly from point clouds.<n>Our 3D encoder achieves zero-shot 3D part segmentation with fast single-pass inference without any test-time multi-view rendering.
  arXiv  Detail & Related papers  (2026-01-05T18:55:45Z)
• RAG-6DPose: Retrieval-Augmented 6D Pose Estimation via Leveraging CAD as Knowledge Base [112.72361202480154]
  We present RAG-6DPose, a retrieval-augmented approach that leverages 3D CAD models as a knowledge base.<n> Experimental results on standard benchmarks and real-world robotic tasks demonstrate the effectiveness and robustness of our approach.
  arXiv  Detail & Related papers  (2025-06-23T17:19:41Z)
• CADCrafter: Generating Computer-Aided Design Models from Unconstrained Images [69.7768227804928]
  CADCrafter is an image-to-parametric CAD model generation framework that trains solely on synthetic textureless CAD data.<n>We introduce a geometry encoder to accurately capture diverse geometric features.<n>Our approach can robustly handle real unconstrained CAD images, and even generalize to unseen general objects.
  arXiv  Detail & Related papers  (2025-04-07T06:01:35Z)
• From 2D CAD Drawings to 3D Parametric Models: A Vision-Language Approach [15.785592359384292]
  We present CAD2Program, a new method for reconstructing 3D parametric models from 2D CAD drawings.<n>We treat the 2D CAD drawing as an image, regardless of its original format, and encode the image with a standard ViT model.<n>On the output side, our method auto-regressively predicts a general-purpose language describing 3D parametric models in text form.
  arXiv  Detail & Related papers  (2024-12-16T15:41:14Z)
• Img2CAD: Conditioned 3D CAD Model Generation from Single Image with Structured Visual Geometry [12.265852643914439]
  We present Img2CAD, the first knowledge that uses 2D image inputs to generate editable parameters. Img2CAD enables seamless integration between AI 3D reconstruction and CAD representation.
  arXiv  Detail & Related papers  (2024-10-04T13:27:52Z)
• Model2Scene: Learning 3D Scene Representation via Contrastive Language-CAD Models Pre-training [105.3421541518582]
  Current successful methods of 3D scene perception rely on the large-scale annotated point cloud. We propose Model2Scene, a novel paradigm that learns free 3D scene representation from Computer-Aided Design (CAD) models and languages. Model2Scene yields impressive label-free 3D object salient detection with an average mAP of 46.08% and 55.49% on the ScanNet and S3DIS datasets, respectively.
  arXiv  Detail & Related papers  (2023-09-29T03:51:26Z)
• CC3D: Layout-Conditioned Generation of Compositional 3D Scenes [49.281006972028194]
  We introduce CC3D, a conditional generative model that synthesizes complex 3D scenes conditioned on 2D semantic scene layouts. Our evaluations on synthetic 3D-FRONT and real-world KITTI-360 datasets demonstrate that our model generates scenes of improved visual and geometric quality.
  arXiv  Detail & Related papers  (2023-03-21T17:59:02Z)
• SECAD-Net: Self-Supervised CAD Reconstruction by Learning Sketch-Extrude Operations [21.000539206470897]
  SECAD-Net is an end-to-end neural network aimed at reconstructing compact and easy-to-edit CAD models. We show superiority over state-of-the-art alternatives including the closely related method for supervised CAD reconstruction.
  arXiv  Detail & Related papers  (2023-03-19T09:26:03Z)
• Patch2CAD: Patchwise Embedding Learning for In-the-Wild Shape Retrieval from a Single Image [58.953160501596805]
  We propose a novel approach towards constructing a joint embedding space between 2D images and 3D CAD models in a patch-wise fashion. Our approach is more robust than state of the art in real-world scenarios without any exact CAD matches.
  arXiv  Detail & Related papers  (2021-08-20T20:58:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.