CADEngBench: It Looks Like CAD, but Does It Work? Evaluating Parametric Design, Assembly Reasoning, and Physics Simulation

• URL: http://arxiv.org/abs/2608.09296v1
• Date: Mon, 10 Aug 2026 08:47:18 GMT
• Title: CADEngBench: It Looks Like CAD, but Does It Work? Evaluating Parametric Design, Assembly Reasoning, and Physics Simulation
• Abstract summary: A CAD model is not engineering-grade merely because it looks correct.<n>We present CADEngBench, a two-track benchmark for these capabilities.
• Score: 1.7032702581423902
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A CAD model is not engineering-grade merely because it looks correct. It must satisfy design requirements, respond predictably to parameter changes, support controlled edits, match a reference structural response under a declared analysis, and connect to other parts through valid joints. We present CADEngBench, a two-track benchmark for these capabilities. CADEngBench-P evaluates 300 parametric parts, each used for one zero-to-CAD task and one functional-editing task (600 tasks in total), through boundary-representation (B-Rep) validity, engineering and DFM checks, parameter-family perturbations, functional editing, and matched linear-static FEA in CalculiX. CADEngBench-A evaluates 150 body pairs through ranked joint retrieval, exact face-and-edge grounding, joint-frame prediction, and kinematic verification. Across eight multimodal, code-capable models, editing supplied CAD is substantially easier than generating it, while complex edits and matched FEA remain difficult. Assembly predictions often locate the relevant region but fail to recover the recorded joint or mating entities. These results show that CAD evaluation must test engineering behavior rather than appearance alone.

Related papers

• MetaRSI / RSI2: A Meta-Recursive Self-Improving System for Recursive Self-Improving Systems Themselves [65.60481414910728]
  Recursive self-improvement lets a system improve the model-building machinery from its own failures, so every later model inherits the gain.<n>Yet RSI has been validated almost exclusively on coding and formal benchmarks such as science QA and mathematics.<n>We argue RSI must next operate across real, diverse scientific, engineering, and meta-scientific domains, not where formal evaluation is merely tractable.<n>We present MetaRSI-v1, where improvement is the scheduled composition of three typed operators over one unified paradigm.
  arXiv  Detail & Related papers  (2026-09-06T05:15:31Z)
• LLM-Aided Design for Manufacturing: A Multi-Agent System for Intent-Preserving Redesign of CAD for Improved Manufacturability [9.129375375221457]
  We introduce autonomous, intent-preserving Design for Manufacturing (DFM) redesign of CAD parts.<n>We instead produce it as a sequence of individually verified design transitions.<n>Iterating review and verified modification compounds edits into parts more complex than one-shot generators reliably produce.
  arXiv  Detail & Related papers  (2026-09-03T18:19:27Z)
• Embodied CAD: Solver-Grounded LLM Agents for Parametric B-Rep Assembly Modeling [7.918861077775763]
  We present Embodied CAD, solver-grounded LLM agents for parametric B-Rep assembly modeling.<n>The framework combines action grammar constraints, deterministic parameter resolution, and solver-derived rewards for supervised warm-up and GRPO-style refinement.<n>We evaluate Embodied CAD on multi-step mechanical, industrial equipment, and mold-oriented assembly tasks using solver-aligned metrics.
  arXiv  Detail & Related papers  (2026-06-30T07:31:01Z)
• UXBench: Measuring the Actionability of LLM-Generated UX Critiques [58.14408744110387]
  We introduce UXBench, a benchmark for evaluating large language models (LLMs) as interaction-grounded UX judges.<n> UXBench comprises local-first runnable web fixtures spanning ten product-surface families, paired with coverage-gated rubric exploration.<n>We evaluate eight frontier models under both an automated repair-lift protocol and a blind human validation study.
  arXiv  Detail & Related papers  (2026-06-15T06:08:39Z)
• PaintBench: Deterministic Evaluation of Precise Visual Editing [50.50559031686293]
  PaintBench is a benchmark for 20 fundamental precise visual editing operations.<n>Across 11 image editing models, we find overall low performance, with the current highest-performing industry leader scoring only 17.1%.<n>Fine-grained benchmark diagnostics show performance degradations induced by scene variations in object count, background complexity, color scheme, and edit-region size.
  arXiv  Detail & Related papers  (2026-05-29T16:01:14Z)
• Prior Availability in Industrial Visual Sim-to-Real: A Review of CAD-Guided and CAD-Unavailable Regimes [0.0]
  Industrial visual sim-to-real often involves a broader mismatch between available evidence and required decisions.<n>This review reframes industrial visual sim-to-real as a domain-gap problem organized by prior availability.
  arXiv  Detail & Related papers  (2026-05-28T21:18:27Z)
• Self-Improving CAD Generation Agents with Finite Element Analysis as Feedback [21.804668679670655]
  We introduce a more industry-native task formulation that requires a model to produce a fully assembled multi-part STEP file from a free-form engineering brief.<n>FEA validation reveals that Codex (GPT-5.5) and Claude Code (Opus-4.7) agents do not produce a single strict-passing artifact in the main first-attempt sweep.<n>On S2O and Fusion360, the same feedback tools improve geometric reconstruction, with GPT-5.5/xhigh rising from 0.444 to 0.592 Box-IoU on S2O and from 0.397 to 0.505 on Fusion360.
  arXiv  Detail & Related papers  (2026-05-17T13:47:38Z)
• BenchCAD: A Comprehensive, Industry-Standard Benchmark for Programmatic CAD [14.284307846349774]
  We present BenchCAD, a unified benchmark for industrial CAD reasoning.<n> BenchCAD contains 17,900 execution-verified CadQuery programs across 106 industrial part families.<n>It evaluates models through visual question answering, code question answering, image-to-code generation, and instruction-guided code editing.
  arXiv  Detail & Related papers  (2026-05-11T17:13:36Z)
• CADCrafter: Generating Computer-Aided Design Models from Unconstrained Images [69.7768227804928]
  CADCrafter is an image-to-parametric CAD model generation framework that trains solely on synthetic textureless CAD data.<n>We introduce a geometry encoder to accurately capture diverse geometric features.<n>Our approach can robustly handle real unconstrained CAD images, and even generalize to unseen general objects.
  arXiv  Detail & Related papers  (2025-04-07T06:01:35Z)
• PHT-CAD: Efficient CAD Parametric Primitive Analysis with Progressive Hierarchical Tuning [52.681829043446044]
  ParaCAD comprises over 10 million annotated drawings for training and 3,000 real-world industrial drawings with complex topological structures and physical constraints for test.<n> PHT-CAD is a novel 2D PPA framework that harnesses the modality alignment and reasoning capabilities of Vision-Language Models.
  arXiv  Detail & Related papers  (2025-03-23T17:24:32Z)
• End-to-End Object Detection with Transformers [88.06357745922716]
  We present a new method that views object detection as a direct set prediction problem. Our approach streamlines the detection pipeline, effectively removing the need for many hand-designed components. The main ingredients of the new framework, called DEtection TRansformer or DETR, are a set-based global loss.
  arXiv  Detail & Related papers  (2020-05-26T17:06:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.