OpenECAD: An Efficient Visual Language Model for Computer-Aided Design
- URL: http://arxiv.org/abs/2406.09913v2
- Date: Sun, 23 Jun 2024 02:22:48 GMT
- Title: OpenECAD: An Efficient Visual Language Model for Computer-Aided Design
- Authors: Zhe Yuan, Jianqi Shi, Yanhong Huang,
- Abstract summary: We fine-tuned pre-trained models to create OpenECAD, leveraging the visual, logical, coding, and general capabilities of visual language models.
OpenECAD can process images of 3D designs as input and generate highly structured 2D sketches and 3D construction commands.
- Score: 1.481550828146527
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Computer-aided design (CAD) tools are utilized in the manufacturing industry for modeling everything from cups to spacecraft. These programs are complex to use and typically require years of training and experience to master. Structured and well-constrained 2D sketches and 3D constructions are crucial components of CAD modeling. A well-executed CAD model can be seamlessly integrated into the manufacturing process, thereby enhancing production efficiency. Deep generative models of 3D shapes and 3D object reconstruction models have garnered significant research interest. However, most of these models are represented in discrete forms. Moreover, the few models based on CAD operations often have substantial input restrictions. In this work, we fine-tuned pre-trained models to create OpenECAD (0.55B, 0.89B, 2.4B, 3.2B and 4.2B), leveraging the visual, logical, coding, and general capabilities of visual language models. OpenECAD can process images of 3D designs as input and generate highly structured 2D sketches and 3D construction commands. These outputs can be directly used with existing CAD tools' APIs to generate project files. To train our network, we created a new CAD dataset. This dataset is based on existing public CAD datasets, with adjustments and augmentations to meet the requirements of VLM training.
Related papers
- PS-CAD: Local Geometry Guidance via Prompting and Selection for CAD Reconstruction [86.726941702182]
We introduce geometric guidance into the reconstruction network PS-CAD.
We provide the geometry of surfaces where the current reconstruction differs from the complete model as a point cloud.
Second, we use geometric analysis to extract a set of planar prompts, that correspond to candidate surfaces.
arXiv Detail & Related papers (2024-05-24T03:43:55Z) - Geometric Deep Learning for Computer-Aided Design: A Survey [85.79012726689511]
This survey offers a comprehensive overview of learning-based methods in computer-aided design.
It includes similarity analysis and retrieval, 2D and 3D CAD model synthesis, and CAD generation from point clouds.
It provides a complete list of benchmark datasets and their characteristics, along with open-source codes that have propelled research in this domain.
arXiv Detail & Related papers (2024-02-27T17:11:35Z) - Model2Scene: Learning 3D Scene Representation via Contrastive
Language-CAD Models Pre-training [105.3421541518582]
Current successful methods of 3D scene perception rely on the large-scale annotated point cloud.
We propose Model2Scene, a novel paradigm that learns free 3D scene representation from Computer-Aided Design (CAD) models and languages.
Model2Scene yields impressive label-free 3D object salient detection with an average mAP of 46.08% and 55.49% on the ScanNet and S3DIS datasets, respectively.
arXiv Detail & Related papers (2023-09-29T03:51:26Z) - SECAD-Net: Self-Supervised CAD Reconstruction by Learning Sketch-Extrude
Operations [21.000539206470897]
SECAD-Net is an end-to-end neural network aimed at reconstructing compact and easy-to-edit CAD models.
We show superiority over state-of-the-art alternatives including the closely related method for supervised CAD reconstruction.
arXiv Detail & Related papers (2023-03-19T09:26:03Z) - AutoCAD: Automatically Generating Counterfactuals for Mitigating
Shortcut Learning [70.70393006697383]
We present AutoCAD, a fully automatic and task-agnostic CAD generation framework.
In this paper, we present AutoCAD, a fully automatic and task-agnostic CAD generation framework.
arXiv Detail & Related papers (2022-11-29T13:39:53Z) - GET3D: A Generative Model of High Quality 3D Textured Shapes Learned
from Images [72.15855070133425]
We introduce GET3D, a Generative model that directly generates Explicit Textured 3D meshes with complex topology, rich geometric details, and high-fidelity textures.
GET3D is able to generate high-quality 3D textured meshes, ranging from cars, chairs, animals, motorbikes and human characters to buildings.
arXiv Detail & Related papers (2022-09-22T17:16:19Z) - CADOps-Net: Jointly Learning CAD Operation Types and Steps from
Boundary-Representations [17.051792180335354]
This paper proposes a new deep neural network, CADOps-Net, that jointly learns the CAD operation types and the decomposition into different CAD operation steps.
Compared to existing datasets, the complexity and variety of CC3D-Ops models are closer to those used for industrial purposes.
arXiv Detail & Related papers (2022-08-22T19:12:20Z) - Using Machine Learning to Predict Engineering Technology Students'
Success with Computer Aided Design [50.591267188664666]
We show how data combined with machine learning techniques can predict how well a particular student will perform in a design task.
We found that our models using early design sequence actions are particularly valuable for prediction.
Further improvements to these models could lead to earlier predictions and thus provide students feedback sooner to enhance their learning.
arXiv Detail & Related papers (2021-08-12T20:24:54Z) - 'CADSketchNet' -- An Annotated Sketch dataset for 3D CAD Model Retrieval
with Deep Neural Networks [0.8155575318208631]
The research work presented in this paper aims at developing a dataset suitable for building a retrieval system for 3D CAD models based on deep learning.
The paper also aims at evaluating the performance of various retrieval system or a search engine for 3D CAD models that accepts a sketch image as the input query.
arXiv Detail & Related papers (2021-07-13T16:10:16Z) - DeepCAD: A Deep Generative Network for Computer-Aided Design Models [37.655225142981564]
We present the first 3D generative model for a drastically different shape representation -- describing a shape as a sequence of computer-aided design (CAD) operations.
Drawing an analogy between CAD operations and natural language, we propose a CAD generative network based on the Transformer.
arXiv Detail & Related papers (2021-05-20T03:29:18Z) - Computer-Aided Design as Language [16.79054488229662]
We propose a machine learning model capable of automatically generating highly structured 2D sketches.
Our method is a combination of a general-purpose language modeling technique alongside an off-the-shelf data serialization protocol.
We show that our approach has enough flexibility to accommodate the complexity of the domain and performs well for both unconditional synthesis and image-to-sketch translation.
arXiv Detail & Related papers (2021-05-06T15:43:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.