BlenderLLM: Training Large Language Models for Computer-Aided Design with Self-improvement

• URL: http://arxiv.org/abs/2412.14203v1
• Date: Mon, 16 Dec 2024 14:34:02 GMT
• Title: BlenderLLM: Training Large Language Models for Computer-Aided Design with Self-improvement
• Authors: Yuhao Du, Shunian Chen, Wenbo Zan, Peizhao Li, Mingxuan Wang, Dingjie Song, Bo Li, Yan Hu, Benyou Wang,
• Abstract summary: We present BlenderLLM, a framework for training Large Language Models (LLMs) in Computer-Aided Design (CAD) Our results reveal that existing models demonstrate significant limitations in generating accurate CAD scripts. Through minimal instruction-based fine-tuning and iterative self-improvement, BlenderLLM significantly surpasses these models in both functionality and accuracy of CAD script generation.
• Score: 45.19076032719869
• License:
• Abstract: The application of Large Language Models (LLMs) in Computer-Aided Design (CAD) remains an underexplored area, despite their remarkable advancements in other domains. In this paper, we present BlenderLLM, a novel framework for training LLMs specifically for CAD tasks leveraging a self-improvement methodology. To support this, we developed a bespoke training dataset, BlendNet, and introduced a comprehensive evaluation suite, CADBench. Our results reveal that existing models demonstrate significant limitations in generating accurate CAD scripts. However, through minimal instruction-based fine-tuning and iterative self-improvement, BlenderLLM significantly surpasses these models in both functionality and accuracy of CAD script generation. This research establishes a strong foundation for the application of LLMs in CAD while demonstrating the transformative potential of self-improving models in advancing CAD automation. We encourage further exploration and adoption of these methodologies to drive innovation in the field. The dataset, model, benchmark, and source code are publicly available at https://github.com/FreedomIntelligence/BlenderLLM

Related papers

• CAD-MLLM: Unifying Multimodality-Conditioned CAD Generation With MLLM [39.113795259823476]
  We introduce the CAD-MLLM, the first system capable of generating parametric CAD models conditioned on the multimodal input. We use advanced large language models (LLMs) to align the feature space across diverse multi-modalities data and CAD models' vectorized representations. Our resulting dataset, named Omni-CAD, is the first multimodal CAD dataset that contains textual description, multi-view images, points, and command sequence for each CAD model.
  arXiv  Detail & Related papers  (2024-11-07T18:31:08Z)
• CadVLM: Bridging Language and Vision in the Generation of Parametric CAD Sketches [24.239470848849418]
  Parametric Computer-Aided Design (CAD) is central to contemporary mechanical design. We propose CadVLM, an end-to-end vision language model for CAD generation.
  arXiv  Detail & Related papers  (2024-09-26T01:22:29Z)
• Apple Intelligence Foundation Language Models [109.60033785567484]
  This report describes the model architecture, the data used to train the model, the training process, and the evaluation results. We highlight our focus on Responsible AI and how the principles are applied throughout the model development.
  arXiv  Detail & Related papers  (2024-07-29T18:38:49Z)
• OpenECAD: An Efficient Visual Language Model for Editable 3D-CAD Design [1.481550828146527]
  We fine-tuned pre-trained models to create OpenECAD models (0.55B, 0.89B, 2.4B and 3.1B) OpenECAD models can process images of 3D designs as input and generate highly structured 2D sketches and 3D construction commands. These outputs can be directly used with existing CAD tools' APIs to generate project files.
  arXiv  Detail & Related papers  (2024-06-14T10:47:52Z)
• Query2CAD: Generating CAD models using natural language queries [6.349503549199403]
  We introduce Query2CAD, a novel framework to generate CAD designs. The framework uses a large language model to generate executable CAD macros. Query2CAD operates without supervised data or additional training, using the LLM as both a generator and a refiner.
  arXiv  Detail & Related papers  (2024-05-31T19:17:00Z)
• Geometric Deep Learning for Computer-Aided Design: A Survey [85.79012726689511]
  This survey offers a comprehensive overview of learning-based methods in computer-aided design. It includes similarity analysis and retrieval, 2D and 3D CAD model synthesis, and CAD generation from point clouds. It provides a complete list of benchmark datasets and their characteristics, along with open-source codes that have propelled research in this domain.
  arXiv  Detail & Related papers  (2024-02-27T17:11:35Z)
• Scaling Vision-Language Models with Sparse Mixture of Experts [128.0882767889029]
  We show that mixture-of-experts (MoE) techniques can achieve state-of-the-art performance on a range of benchmarks over dense models of equivalent computational cost. Our research offers valuable insights into stabilizing the training of MoE models, understanding the impact of MoE on model interpretability, and balancing the trade-offs between compute performance when scaling vision-language models.
  arXiv  Detail & Related papers  (2023-03-13T16:00:31Z)
• AutoCAD: Automatically Generating Counterfactuals for Mitigating Shortcut Learning [70.70393006697383]
  We present AutoCAD, a fully automatic and task-agnostic CAD generation framework. In this paper, we present AutoCAD, a fully automatic and task-agnostic CAD generation framework.
  arXiv  Detail & Related papers  (2022-11-29T13:39:53Z)
• How Does Counterfactually Augmented Data Impact Models for Social Computing Constructs? [35.29235215101502]
  We investigate the benefits of counterfactually augmented data (CAD) for social NLP models by focusing on three social computing constructs -- sentiment, sexism, and hate speech. We find that while models trained on CAD show lower in-domain performance, they generalize better out-of-domain.
  arXiv  Detail & Related papers  (2021-09-14T23:46:39Z)
• Quantitatively Assessing the Benefits of Model-driven Development in Agent-based Modeling and Simulation [80.49040344355431]
  This paper compares the use of MDD and ABMS platforms in terms of effort and developer mistakes. The obtained results show that MDD4ABMS requires less effort to develop simulations with similar (sometimes better) design quality than NetLogo.
  arXiv  Detail & Related papers  (2020-06-15T23:29:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.