OmniCAD: A Large-Scale Benchmark for 3D Spatial Reasoning in Robotics Assemblies

• URL: http://arxiv.org/abs/2608.22637v1
• Date: Sun, 23 Aug 2026 22:42:28 GMT
• Title: OmniCAD: A Large-Scale Benchmark for 3D Spatial Reasoning in Robotics Assemblies
• Abstract summary: We introduce OmniCAD, a large-scale benchmark for assembly-aware 3D spatial reasoning across diverse industrial systems.<n> OmniCAD contains 25k mechanical assemblies, with an average of 12 parts per assembly and 21 types of mate relationships.<n>We will open-source the benchmark, evaluation code, and tool interfaces to support research on accurate, physically valid, and scalable 3D assembly reasoning.
• Score: 12.751658252598366
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent vision-language models (VLMs) show strong capabilities in robotic perception and spatial reasoning, yet their ability to reason about complex mechanical assemblies remains underexplored. We introduce OmniCAD, a large-scale benchmark for assembly-aware 3D spatial reasoning across diverse industrial systems, including robotic mechanisms, automotive components, aerospace structures, and agricultural machinery. OmniCAD contains 25k mechanical assemblies, with an average of 12 parts per assembly and 21 types of mate relationships. Each assembly includes a human-verified ground-truth 3D model and renderings from 20 viewpoints. The benchmark evaluates three capabilities: (1) component-level 3D spatial reasoning, requiring prediction of part positions and orientations; (2) part-to-part relational reasoning, requiring identification of mating relationships and assembly constraints; and (3) tool-augmented agentic reasoning, where models iteratively select viewpoints, inspect visual evidence, and refine predictions. Experiments show that current VLMs struggle with industrial assembly reasoning, often producing inaccurate poses, invalid mating relationships, part interpenetration, and degraded performance as assembly complexity increases. We will open-source the benchmark, evaluation code, and tool interfaces to support research on accurate, physically valid, and scalable 3D assembly reasoning.

Related papers

• OmniMech: All-in-one Multimodal Mechanical Benchmark for 3D Reconstruction [15.427717696810456]
  We introduce OmniMech, the first million-scale benchmark for evaluating vision-language models on CAD generation from industrial data.<n> OmniMech contains more than 251,000 fully dimensioned and toleranced 2D orthographic drawings, paired with native CAD models.<n>We will release the benchmark data, evaluation code, and tool interfaces to support future research.
  arXiv  Detail & Related papers  (2026-08-06T02:36:22Z)
• ViewMind3D: Modular View-Aware Inference for Training-Free 3D-QA [26.947027118604368]
  textbfViewMind3D is a fully training-free and modular framework for 3D spatial reasoning over multi-view observations of a scene.<n>We show that ViewMind3D achieves competitive performance compared to prior training-free and fine-tuned 3D-LLMs.
  arXiv  Detail & Related papers  (2026-07-30T16:14:30Z)
• Embodied3DBench: Benchmarking Low-Level Embodied Spatial Intelligence of Vision Language Models [50.6415287154632]
  Embodied3DBench is a robot-centric benchmark targeting low-level spatial intelligence in 3D environments.<n>The benchmark spans 12 subcategories and contains over 21k high-quality question-answer pairs.
  arXiv  Detail & Related papers  (2026-05-27T20:28:56Z)
• PointACT: Vision-Language-Action Models with Multi-Scale Point-Action Interaction [57.63073414949329]
  We propose PointACT, a dual-system 3D-aware VLA policy that integrates hierarchical 3D point cloud representations directly into the action decoding process.<n>PointACT employs a multi-scale point-action interaction mechanism with efficient bottleneck window self-attention, enabling evolving action tokens to densely attend to both local geometric detail and global scene structure.
  arXiv  Detail & Related papers  (2026-05-20T17:10:31Z)
• AssemLM: Spatial Reasoning Multimodal Large Language Models for Robotic Assembly [45.963541758601274]
  We propose AssemLM, a spatial multimodal large language model tailored for robotic assembly.<n>AssemLM integrates assembly manuals, point clouds, and textual instructions to reason about and predict task-critical 6D assembly poses.<n>Our model achieves state-of-the-art performance in 6D pose reasoning across diverse assembly scenarios.
  arXiv  Detail & Related papers  (2026-04-10T05:43:39Z)
• From Perception to Action: An Interactive Benchmark for Vision Reasoning [51.11355591375073]
  Causal Hierarchy of Actions and Interactions (CHAIN) benchmark designed to evaluate whether models can understand, plan, and execute structured action sequences grounded in physical constraints.<n> CHAIN shifts evaluation from passive perception to active problem solving, spanning tasks such as interlocking mechanical puzzles and 3D stacking and packing.<n>Our results show that top-performing models still struggle to internalize physical structure and causal constraints, often failing to produce reliable long-horizon plans and cannot robustly translate perceived structure into effective actions.
  arXiv  Detail & Related papers  (2026-02-24T15:33:02Z)
• Context-Aware Mapping of 2D Drawing Annotations to 3D CAD Features Using LLM-Assisted Reasoning for Manufacturing Automation [0.05090720572281118]
  This paper presents a deterministic-first, context-aware framework that maps 2D drawing entities to 3D CAD features.<n> Experiments on 20 real CAD-drawing pairs achieve a mean precision of 83.67%, recall of 90.46%, and F1 score of 86.29%.
  arXiv  Detail & Related papers  (2026-02-20T15:46:57Z)
• Large Language Models and 3D Vision for Intelligent Robotic Perception and Autonomy [1.7523719472700858]
  Large Language Models (LLMs) with 3D vision is emerging as a transformative approach to enhancing robotic sensing technologies.<n>This review provides a comprehensive analysis of state-of-the-art methodologies, applications and challenges at the intersection of LLMs and 3D vision.
  arXiv  Detail & Related papers  (2025-11-14T11:32:25Z)
• SURPRISE3D: A Dataset for Spatial Understanding and Reasoning in Complex 3D Scenes [105.8644620467576]
  We introduce Stextscurprise3D, a novel dataset designed to evaluate language-guided spatial reasoning segmentation in complex 3D scenes.<n>Stextscurprise3D consists of more than 200k vision language pairs across 900+ detailed indoor scenes from ScanNet++ v2.<n>The dataset contains 89k+ human-annotated spatial queries deliberately crafted without object name.
  arXiv  Detail & Related papers  (2025-07-10T14:01:24Z)
• Articulate-Anything: Automatic Modeling of Articulated Objects via a Vision-Language Foundation Model [35.184607650708784]
  Articulate-Anything automates the articulation of diverse, complex objects from many input modalities, including text, images, and videos.<n>Our system exploits existing 3D asset datasets via a mesh retrieval mechanism, along with an actor-critic system that iteratively proposes, evaluates, and refines solutions.
  arXiv  Detail & Related papers  (2024-10-03T19:42:16Z)
• MetaGraspNet: A Large-Scale Benchmark Dataset for Vision-driven Robotic Grasping via Physics-based Metaverse Synthesis [78.26022688167133]
  We present a large-scale benchmark dataset for vision-driven robotic grasping via physics-based metaverse synthesis. The proposed dataset contains 100,000 images and 25 different object types. We also propose a new layout-weighted performance metric alongside the dataset for evaluating object detection and segmentation performance.
  arXiv  Detail & Related papers  (2021-12-29T17:23:24Z)
• MoreFusion: Multi-object Reasoning for 6D Pose Estimation from Volumetric Fusion [19.034317851914725]
  We present a system which can estimate the accurate poses of multiple known objects in contact and occlusion from real-time, embodied multi-view vision. Our approach makes 3D object pose proposals from single RGB-D views, accumulates pose estimates and non-parametric occupancy information from multiple views as the camera moves. We verify the accuracy and robustness of our approach experimentally on 2 object datasets: YCB-Video, and our own challenging Cluttered YCB-Video.
  arXiv  Detail & Related papers  (2020-04-09T02:29:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.