Novel class discovery meets foundation models for 3D semantic segmentation

• URL: http://arxiv.org/abs/2312.03782v2
• Date: Tue, 20 Aug 2024 09:13:35 GMT
• Title: Novel class discovery meets foundation models for 3D semantic segmentation
• Abstract summary: This paper introduces the novel task of NCD for point cloud semantic segmentation. It demonstrates that directly transposing the only existing NCD method for 2D semantic segmentation to 3D data yields suboptimal results. A new NCD approach based on online clustering, uncertainty estimation, and semantic distillation is presented.
• Score: 21.879254503403516
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The task of Novel Class Discovery (NCD) in semantic segmentation entails training a model able to accurately segment unlabelled (novel) classes, relying on the available supervision from annotated (base) classes. Although extensively investigated in 2D image data, the extension of the NCD task to the domain of 3D point clouds represents a pioneering effort, characterized by assumptions and challenges that are not present in the 2D case. This paper represents an advancement in the analysis of point cloud data in four directions. Firstly, it introduces the novel task of NCD for point cloud semantic segmentation. Secondly, it demonstrates that directly transposing the only existing NCD method for 2D image semantic segmentation to 3D data yields suboptimal results. Thirdly, a new NCD approach based on online clustering, uncertainty estimation, and semantic distillation is presented. Lastly, a novel evaluation protocol is proposed to rigorously assess the performance of NCD in point cloud semantic segmentation. Through comprehensive evaluations on the SemanticKITTI, SemanticPOSS, and S3DIS datasets, the paper demonstrates substantial superiority of the proposed method over the considered baselines.

Related papers

• P-SLCR: Unsupervised Point Cloud Semantic Segmentation via Prototypes Structure Learning and Consistent Reasoning [10.853799342986177]
  Current semantic segmentation approaches for point cloud scenes rely on manual labeling.<n>Unsupervised point cloud learning poses significant challenges due to the absence of annotation information.<n>We propose a novel prototype library-driven unsupervised point cloud semantic segmentation strategy.
  arXiv  Detail & Related papers  (2026-03-06T14:23:10Z)
• CP-VoteNet: Contrastive Prototypical VoteNet for Few-Shot Point Cloud Object Detection [7.205000222081269]
  Few-shot point cloud 3D object detection (FS3D) aims to identify and localise objects of novel classes from point clouds. We introduce contrastive semantics mining, which enables the network to extract discriminative categorical features. Through refined primitive geometric structures, the transferability of feature encoding from base to novel classes is significantly enhanced.
  arXiv  Detail & Related papers  (2024-08-30T06:13:49Z)
• Multi-modality Affinity Inference for Weakly Supervised 3D Semantic Segmentation [47.81638388980828]
  We propose a simple yet effective scene-level weakly supervised point cloud segmentation method with a newly introduced multi-modality point affinity inference module. Our method outperforms the state-of-the-art by 4% to 6% mIoU on the ScanNet and S3DIS benchmarks.
  arXiv  Detail & Related papers  (2023-12-27T14:01:35Z)
• Novel Class Discovery for 3D Point Cloud Semantic Segmentation [13.024681120553906]
  Novel class discovery (NCD) for semantic segmentation is the task of learning a model that can segment unlabelled (novel) classes using only the supervision from labelled (base) classes. This paper is presented to advance the state of the art on point cloud data analysis in four directions.
  arXiv  Detail & Related papers  (2023-03-21T06:10:39Z)
• Dual Adaptive Transformations for Weakly Supervised Point Cloud Segmentation [78.6612285236938]
  We propose a novel DAT (textbfDual textbfAdaptive textbfTransformations) model for weakly supervised point cloud segmentation. We evaluate our proposed DAT model with two popular backbones on the large-scale S3DIS and ScanNet-V2 datasets.
  arXiv  Detail & Related papers  (2022-07-19T05:43:14Z)
• Guided Point Contrastive Learning for Semi-supervised Point Cloud Semantic Segmentation [90.2445084743881]
  We present a method for semi-supervised point cloud semantic segmentation to adopt unlabeled point clouds in training to boost the model performance. Inspired by the recent contrastive loss in self-supervised tasks, we propose the guided point contrastive loss to enhance the feature representation and model generalization ability.
  arXiv  Detail & Related papers  (2021-10-15T16:38:54Z)
• Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception [122.53774221136193]
  State-of-the-art methods for driving-scene LiDAR-based perception often project the point clouds to 2D space and then process them via 2D convolution. A natural remedy is to utilize the 3D voxelization and 3D convolution network. We propose a new framework for the outdoor LiDAR segmentation, where cylindrical partition and asymmetrical 3D convolution networks are designed to explore the 3D geometric pattern.
  arXiv  Detail & Related papers  (2021-09-12T06:25:11Z)
• PC-RGNN: Point Cloud Completion and Graph Neural Network for 3D Object Detection [57.49788100647103]
  LiDAR-based 3D object detection is an important task for autonomous driving. Current approaches suffer from sparse and partial point clouds of distant and occluded objects. In this paper, we propose a novel two-stage approach, namely PC-RGNN, dealing with such challenges by two specific solutions.
  arXiv  Detail & Related papers  (2020-12-18T18:06:43Z)
• Complete & Label: A Domain Adaptation Approach to Semantic Segmentation of LiDAR Point Clouds [49.47017280475232]
  We study an unsupervised domain adaptation problem for the semantic labeling of 3D point clouds. We take a Complete and Label approach to recover the underlying surfaces before passing them to a segmentation network. The recovered 3D surfaces serve as a canonical domain, from which semantic labels can transfer across different LiDAR sensors.
  arXiv  Detail & Related papers  (2020-07-16T17:42:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.