CDIS: Cross-Dimensional Class-Agnostic 3D Instance Segmentation via 2D Mask Tracking and 3D-2D Projection Merging

• URL: http://arxiv.org/abs/2607.17778v1
• Date: Mon, 20 Jul 2026 10:09:12 GMT
• Title: CDIS: Cross-Dimensional Class-Agnostic 3D Instance Segmentation via 2D Mask Tracking and 3D-2D Projection Merging
• Abstract summary: Class-agnostic 3D instance segmentation is critical for robotic systems operating in unknown environments.<n>We introduce Cross-Dimensional Class-Agnostic 3D Instance (CDIS), a zero-shot framework that explicitly tracks 2D instance masks across frames and associates them with 3D superpoints.<n>This cross-dimensional reasoning links temporally stable 2D tracks with spatially coherent 3D regions, producing globally consistent 3D instance labels without any 3D-specific training.
• Score: 24.37584731810519
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Class-agnostic 3D instance segmentation is critical for robotic systems operating in unknown environments, enabling perception of previously unseen objects for reliable manipulation and navigation. Existing approaches typically project per-frame 2D instance masks into 3D and merge them, which often breaks object identities across time and yields fragmented 3D instances. We introduce Cross-Dimensional Class-Agnostic 3D Instance Segmentation (CDIS), a zero-shot framework that explicitly tracks 2D instance masks across frames and associates them with 3D superpoints, creating a feedback loop between 2D and 3D. This cross-dimensional reasoning links temporally stable 2D tracks with spatially coherent 3D regions, producing globally consistent 3D instance labels without any 3D-specific training. Experiments on benchmark datasets demonstrate that CDIS achieves higher accuracy and consistency than state-of-the-art zero-shot methods, while remaining efficient and scalable to diverse real-world environments.

Related papers

• MV3DIS: Multi-View Mask Matching via 3D Guides for Zero-Shot 3D Instance Segmentation [12.513474673735963]
  Conventional 3D instance segmentation methods rely on labor-intensive 3D annotations for supervised training.<n>Recent approaches leverage multi-view 2D masks from the Segment Anything Model (SAM) to guide the merging of 3D geometric primitives.<n>We propose MV3DIS, a coarse-to-fine framework for zero-shot 3D instance segmentation that explicitly incorporates 3D priors.
  arXiv  Detail & Related papers  (2026-04-10T03:26:52Z)
• Class-agnostic 3D Segmentation by Granularity-Consistent Automatic 2D Mask Tracking [10.223105883919278]
  We introduce a Granularity-Consistent automatic 2D Mask Tracking approach that maintains temporal correspondences across frames.<n>Our method effectively generated consistent and accurate 3D segmentations.
  arXiv  Detail & Related papers  (2025-11-02T03:52:42Z)
• SegDINO3D: 3D Instance Segmentation Empowered by Both Image-Level and Object-Level 2D Features [61.13570953713554]
  SegDINO3D is a novel Transformer encoder-decoder framework for 3D instance segmentation.<n>It fully leverage 2D representation from a pre-trained 2D detection model, including both image-level and object-level features.<n>SegDINO3D achieves the state-of-the-art performance on the ScanNetV2 and ScanNet200 3D instance segmentation benchmarks.
  arXiv  Detail & Related papers  (2025-09-19T15:41:10Z)
• Integrating SAM Supervision for 3D Weakly Supervised Point Cloud Segmentation [66.65719382619538]
  Current methods for 3D semantic segmentation propose training models with limited annotations to address the difficulty of annotating large, irregular, and unordered 3D point cloud data.<n>We present a novel approach that maximizes the utility of sparsely available 3D annotations incorporating segmentation masks generated by 2D foundation models.
  arXiv  Detail & Related papers  (2025-08-27T14:13:01Z)
• OnlineAnySeg: Online Zero-Shot 3D Segmentation by Visual Foundation Model Guided 2D Mask Merging [36.9859733771263]
  We propose an efficient method for lifting 2D masks into a unified 3D instance using a hashing technique.<n>By employing voxel hashing for efficient 3D scene querying, our approach reduces the time complexity of costly spatial overlap queries.<n>Our approach achieves state-of-the-art performance in online, zero-shot 3D instance segmentation with leading efficiency.
  arXiv  Detail & Related papers  (2025-03-03T08:48:06Z)
• DCSEG: Decoupled 3D Open-Set Segmentation using Gaussian Splatting [0.0]
  We present a decoupled 3D segmentation pipeline to ensure modularity and adaptability to novel 3D representations.<n>We evaluate on synthetic and real-world indoor datasets, demonstrating improved performance over comparable NeRF-based pipelines.
  arXiv  Detail & Related papers  (2024-12-14T21:26:44Z)
• Bootstraping Clustering of Gaussians for View-consistent 3D Scene Understanding [59.51535163599723]
  FreeGS is an unsupervised semantic-embedded 3DGS framework that achieves view-consistent 3D scene understanding without the need for 2D labels.<n>FreeGS performs comparably to state-of-the-art methods while avoiding the complex data preprocessing workload.
  arXiv  Detail & Related papers  (2024-11-29T08:52:32Z)
• Any3DIS: Class-Agnostic 3D Instance Segmentation by 2D Mask Tracking [6.599971425078935]
  Existing 3D instance segmentation methods frequently encounter issues with over-segmentation, leading to redundant and inaccurate 3D proposals that complicate downstream tasks. This challenge arises from their unsupervised merging approach, where dense 2D masks are lifted across frames into point clouds to form 3D candidate proposals without direct supervision. We propose a 3D-Aware 2D Mask Tracking module that uses robust 3D priors from a 2D mask segmentation and tracking foundation model (SAM-2) to ensure consistent object masks across video frames.
  arXiv  Detail & Related papers  (2024-11-25T08:26:31Z)
• SAM-guided Graph Cut for 3D Instance Segmentation [60.75119991853605]
  This paper addresses the challenge of 3D instance segmentation by simultaneously leveraging 3D geometric and multi-view image information. We introduce a novel 3D-to-2D query framework to effectively exploit 2D segmentation models for 3D instance segmentation. Our method achieves robust segmentation performance and can generalize across different types of scenes.
  arXiv  Detail & Related papers  (2023-12-13T18:59:58Z)
• Segment Anything in 3D with Radiance Fields [83.14130158502493]
  This paper generalizes the Segment Anything Model (SAM) to segment 3D objects. We refer to the proposed solution as SA3D, short for Segment Anything in 3D. We show in experiments that SA3D adapts to various scenes and achieves 3D segmentation within seconds.
  arXiv  Detail & Related papers  (2023-04-24T17:57:15Z)
• Joint-MAE: 2D-3D Joint Masked Autoencoders for 3D Point Cloud Pre-training [65.75399500494343]
  Masked Autoencoders (MAE) have shown promising performance in self-supervised learning for 2D and 3D computer vision. We propose Joint-MAE, a 2D-3D joint MAE framework for self-supervised 3D point cloud pre-training.
  arXiv  Detail & Related papers  (2023-02-27T17:56:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.