CAD-Prompted SAM3: Geometry-Conditioned Instance Segmentation for Industrial Objects

• URL: http://arxiv.org/abs/2602.20551v1
• Date: Tue, 24 Feb 2026 05:10:22 GMT
• Title: CAD-Prompted SAM3: Geometry-Conditioned Instance Segmentation for Industrial Objects
• Abstract summary: We propose a CAD-prompted segmentation framework built on SAM3.<n>It uses canonical multi-view renderings of a CAD model as prompt input.<n>The rendered views provide geometry-based conditioning independent of surface appearance.
• Score: 8.676167989526347
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Verbal-prompted segmentation is inherently limited by the expressiveness of natural language and struggles with uncommon, instance-specific, or difficult-to-describe objects: scenarios frequently encountered in manufacturing and 3D printing environments. While image exemplars provide an alternative, they primarily encode appearance cues such as color and texture, which are often unrelated to a part's geometric identity. In industrial settings, a single component may be produced in different materials, finishes, or colors, making appearance-based prompting unreliable. In contrast, such objects are typically defined by precise CAD models that capture their canonical geometry. We propose a CAD-prompted segmentation framework built on SAM3 that uses canonical multi-view renderings of a CAD model as prompt input. The rendered views provide geometry-based conditioning independent of surface appearance. The model is trained using synthetic data generated from mesh renderings in simulation under diverse viewpoints and scene contexts. Our approach enables single-stage, CAD-prompted mask prediction, extending promptable segmentation to objects that cannot be robustly described by language or appearance alone.

Related papers

• G$^2$TAM: Geometry Grounded Track Anything Model [49.05553245076823]
  We propose a unified framework for promptable instance tracking in 3D using only unordered RGB images or videos.<n>G$2$TAM employs spatially aligned geometric representations as implicit memory, ensuring stable instance identity and localization across frames and views.<n>At its core is a cross-modal spatial encoder that integrates visual and textual prompts into a shared geometric space, enabling end-to-end spatial reconstruction and instance-consistent mask prediction.
  arXiv  Detail & Related papers  (2026-07-04T09:32:13Z)
• MV-GEL: Language-Driven Multi-View Geometric Entity Localization on Meshes [0.0]
  MV-GEL is a framework for localizing fine-grained geometric entities on polygon meshes from natural language queries.<n>GELviews is a prompt-conditioned ranking module that prioritizes viewpoints based on language prompted observability of geometric CAD entities.<n>Experiments show up to a 1.7X improvement in face-level IoU and over 4.5X gains in edge-level F1 compared to vanilla baselines.
  arXiv  Detail & Related papers  (2026-06-30T11:46:25Z)
• SCAPO: Self-Supervised Category-Level Articulated Pose Estimation from a Single 3D Observation [54.98864436812187]
  We propose a self-supervised framework that estimates canonical geometry, rigid part segmentation, and joint pivots, axes, and articulation states from a single RGB-D observation.<n> Experiments on synthetic and real articulated-object datasets show that our SCAPO recovers consistent part structure and accurate articulation parameters and outperforms all self-supervised baselines.
  arXiv  Detail & Related papers  (2026-06-01T09:04:49Z)
• Prior Availability in Industrial Visual Sim-to-Real: A Review of CAD-Guided and CAD-Unavailable Regimes [0.0]
  Industrial visual sim-to-real often involves a broader mismatch between available evidence and required decisions.<n>This review reframes industrial visual sim-to-real as a domain-gap problem organized by prior availability.
  arXiv  Detail & Related papers  (2026-05-28T21:18:27Z)
• DreamCAD: Scaling Multi-modal CAD Generation using Differentiable Parametric Surfaces [67.45867367326105]
  We propose DreamCAD, a multi-modal generative framework that directly produces editable BReps from point-level supervision.<n>DreamCAD represents each BRep as a set of parametric patches and uses a differentiable tessellation method to generate meshes.<n>DreamCAD achieves state-of-the-art performance on ABC and averse benchmarks across text, image, and point modalities.
  arXiv  Detail & Related papers  (2026-03-05T19:04:07Z)
• CADKnitter: Compositional CAD Generation from Text and Geometry Guidance [8.644079160190175]
  We propose CADKnitter, a compositional CAD generation framework with a geometry-guided diffusion sampling strategy.<n>CADKnitter is able to generate a complementary CAD part that follows both the geometric constraints of the given CAD model and the semantic constraints of the desired design text prompt.<n>We also curate a dataset, so-called KnitCAD, containing over 310,000 samples of CAD models, along with textual prompts and assembly metadata.
  arXiv  Detail & Related papers  (2025-12-12T01:06:38Z)
• HistCAD: Geometrically Constrained Parametric History-based CAD Dataset [7.7008607520955]
  HistCAD is a large-scale dataset featuring constraint-aware modeling sequences.<n>HistCAD provides a unified benchmark for advancing editable, constraint-aware, and semantically enriched generative CAD modeling.
  arXiv  Detail & Related papers  (2025-12-08T05:52:14Z)
• CADCrafter: Generating Computer-Aided Design Models from Unconstrained Images [69.7768227804928]
  CADCrafter is an image-to-parametric CAD model generation framework that trains solely on synthetic textureless CAD data.<n>We introduce a geometry encoder to accurately capture diverse geometric features.<n>Our approach can robustly handle real unconstrained CAD images, and even generalize to unseen general objects.
  arXiv  Detail & Related papers  (2025-04-07T06:01:35Z)
• Shape from Semantics: 3D Shape Generation from Multi-View Semantics [30.969299308083723]
  Existing 3D reconstruction methods utilize guidances such as 2D images, 3D point clouds, shape contours and single semantics to recover the 3D surface.<n>We propose a novel 3D modeling task called Shape from Semantics'', which aims to create 3D models whose geometry and appearance are consistent with the given text semantics when viewed from different views.
  arXiv  Detail & Related papers  (2025-02-01T07:51:59Z)
• 3D Part Segmentation via Geometric Aggregation of 2D Visual Features [57.20161517451834]
  Supervised 3D part segmentation models are tailored for a fixed set of objects and parts, limiting their transferability to open-set, real-world scenarios.<n>Recent works have explored vision-language models (VLMs) as a promising alternative, using multi-view rendering and textual prompting to identify object parts.<n>To address these limitations, we propose COPS, a COmprehensive model for Parts that blends semantics extracted from visual concepts and 3D geometry to effectively identify object parts.
  arXiv  Detail & Related papers  (2024-12-05T15:27:58Z)
• Sparse Multi-Object Render-and-Compare [33.97243145891282]
  Reconstructing 3D shape and pose of static objects from a single image is an essential task for various industries. Directly predicting 3D shapes produces unrealistic, overly smoothed or tessellated shapes. Retrieving CAD models ensures realistic shapes but requires robust and accurate alignment.
  arXiv  Detail & Related papers  (2023-10-17T12:01:32Z)
• Zero-Shot Text-Guided Object Generation with Dream Fields [111.06026544180398]
  We combine neural rendering with multi-modal image and text representations to synthesize diverse 3D objects. Our method, Dream Fields, can generate the geometry and color of a wide range of objects without 3D supervision. In experiments, Dream Fields produce realistic, multi-view consistent object geometry and color from a variety of natural language captions.
  arXiv  Detail & Related papers  (2021-12-02T17:53:55Z)
• Patch2CAD: Patchwise Embedding Learning for In-the-Wild Shape Retrieval from a Single Image [58.953160501596805]
  We propose a novel approach towards constructing a joint embedding space between 2D images and 3D CAD models in a patch-wise fashion. Our approach is more robust than state of the art in real-world scenarios without any exact CAD matches.
  arXiv  Detail & Related papers  (2021-08-20T20:58:52Z)
• Reconstructing Interactive 3D Scenes by Panoptic Mapping and CAD Model Alignments [81.38641691636847]
  We rethink the problem of scene reconstruction from an embodied agent's perspective. We reconstruct an interactive scene using RGB-D data stream. This reconstructed scene replaces the object meshes in the dense panoptic map with part-based articulated CAD models.
  arXiv  Detail & Related papers  (2021-03-30T05:56:58Z)
• Mask2CAD: 3D Shape Prediction by Learning to Segment and Retrieve [54.054575408582565]
  We propose to leverage existing large-scale datasets of 3D models to understand the underlying 3D structure of objects seen in an image. We present Mask2CAD, which jointly detects objects in real-world images and for each detected object, optimize for the most similar CAD model and its pose. This produces a clean, lightweight representation of the objects in an image.
  arXiv  Detail & Related papers  (2020-07-26T00:08:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.