TOOLCAD: Exploring Tool-Using Large Language Models in Text-to-CAD Generation with Reinforcement Learning

• URL: http://arxiv.org/abs/2604.07960v1
• Date: Thu, 09 Apr 2026 08:22:46 GMT
• Title: TOOLCAD: Exploring Tool-Using Large Language Models in Text-to-CAD Generation with Reinforcement Learning
• Abstract summary: Large Language Models (LLMs) have shown remarkable advancements in enabling language agents to tackle real-world tasks.<n>We propose ToolCAD, a novel agentic CAD framework deploying LLMs as tool-using agents for text-to-CAD generation.
• Score: 4.1430698134829695
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Computer-Aided Design (CAD) is an expert-level task that relies on long-horizon reasoning and coherent modeling actions. Large Language Models (LLMs) have shown remarkable advancements in enabling language agents to tackle real-world tasks. Notably, there has been no investigation into how tool-using LLMs optimally interact with CAD engines, hindering the emergence of LLM-based agentic text-to-CAD modeling systems. We propose ToolCAD, a novel agentic CAD framework deploying LLMs as tool-using agents for text-to-CAD generation. Furthermore, we introduce an interactive CAD modeling gym to rollout reasoning and tool-augmented interaction trajectories with the CAD engine, incorporating hybrid feedback and human supervision. Meanwhile, an end-to-end post-training strategy is presented to enable the LLM agent to elicit refined CAD Modeling Chain of Thought (CAD-CoT) and evolve into proficient CAD tool-using agents via online curriculum reinforcement learning. Our findings demonstrate ToolCAD fills the gap in adopting and training open-source LLMs for CAD tool-using agents, enabling them to perform comparably to proprietary models, paving the way for more accessible and robust autonomous text-to-CAD modeling systems.

Related papers

• CADFS: A Big CAD Program Dataset and Framework for Computer-Aided Design with Large Language Models [80.47087401394882]
  We introduce CADFS, a data-centric framework that enables large vision-language models to generate complex CAD design histories.<n>We construct a dataset of 450k real-world CAD models spanning 15 modeling operations.<n>Fine-tuning a VLM on this representation yields state-of-the-art results in text-conditioned CAD generation and image-based reconstruction.
  arXiv  Detail & Related papers  (2026-05-03T15:12:36Z)
• Towards High-Fidelity CAD Generation via LLM-Driven Program Generation and Text-Based B-Rep Primitive Grounding [18.983809090879348]
  We present FutureCAD, a novel text-to-CAD framework for high-fidelity CAD generation.<n>Our method generates executable CadQuery scripts, and introduces a text-based query mechanism.<n>Experiments show that FutureCADGround achieves state-of-the-art CAD generation performance.
  arXiv  Detail & Related papers  (2026-03-12T11:54:29Z)
• From Intent to Execution: Multimodal Chain-of-Thought Reinforcement Learning for Precise CAD Code Generation [47.67703214044401]
  We propose CAD-RL, a multimodal Chain-of-Thought guided reinforcement learning framework for CAD modeling code generation.<n>Our method combines Cold Start with goal-driven reinforcement learning post training using three task-specific rewards.<n>Experiments demonstrate that CAD-RL achieves significant improvements in reasoning quality, output precision, and code executability.
  arXiv  Detail & Related papers  (2025-08-13T18:30:49Z)
• Generative AI for CAD Automation: Leveraging Large Language Models for 3D Modelling [31.94035963354055]
  Large Language Models (LLMs) are revolutionizing industries by enhancing efficiency, scalability, and innovation.<n>This paper investigates the potential of LLMs in automating Computer-Aided Design (CAD) by integrating FreeCAD with LLM as CAD design tool.<n>We propose a framework where LLMs generate initial CAD scripts from natural language descriptions, which are then executed and refined iteratively based on error feedback.
  arXiv  Detail & Related papers  (2025-07-05T23:30:17Z)
• UniCAD: Efficient and Extendable Architecture for Multi-Task Computer-Aided Diagnosis System [48.83716673786449]
  We propose UniCAD, a unified architecture that seamlessly handles both 2D and 3D medical images.<n>A low-rank adaptation strategy is employed to adapt a pre-trained visual model to the medical image domain, achieving performance on par with fully fine-tuned counterparts.<n>Building on this unified CAD architecture, we establish an open-source platform where researchers can share and access lightweight CAD experts.
  arXiv  Detail & Related papers  (2025-05-14T06:21:27Z)
• CADCrafter: Generating Computer-Aided Design Models from Unconstrained Images [69.7768227804928]
  CADCrafter is an image-to-parametric CAD model generation framework that trains solely on synthetic textureless CAD data.<n>We introduce a geometry encoder to accurately capture diverse geometric features.<n>Our approach can robustly handle real unconstrained CAD images, and even generalize to unseen general objects.
  arXiv  Detail & Related papers  (2025-04-07T06:01:35Z)
• CAD-Assistant: Tool-Augmented VLLMs as Generic CAD Task Solvers [20.440350517523314]
  CAD-Assistant addresses user queries by generating actions that are iteratively executed on a Python interpreter equipped with the FreeCAD software.<n>We consider a wide range of CAD-specific tools including a sketch image parameterizer, rendering modules, a 2D cross-section generator, and other specialized routines.
  arXiv  Detail & Related papers  (2024-12-18T12:57:56Z)
• BlenderLLM: Training Large Language Models for Computer-Aided Design with Self-improvement [45.19076032719869]
  We present BlenderLLM, a framework for training Large Language Models (LLMs) in Computer-Aided Design (CAD)<n>Our results reveal that existing models demonstrate significant limitations in generating accurate CAD scripts.<n>Through minimal instruction-based fine-tuning and iterative self-improvement, BlenderLLM significantly surpasses these models in both functionality and accuracy of CAD script generation.
  arXiv  Detail & Related papers  (2024-12-16T14:34:02Z)
• AutoCAD: Automatically Generating Counterfactuals for Mitigating Shortcut Learning [70.70393006697383]
  We present AutoCAD, a fully automatic and task-agnostic CAD generation framework. In this paper, we present AutoCAD, a fully automatic and task-agnostic CAD generation framework.
  arXiv  Detail & Related papers  (2022-11-29T13:39:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.