Number-Adaptive Prototype Learning for 3D Point Cloud Semantic Segmentation

• URL: http://arxiv.org/abs/2210.09948v1
• Date: Tue, 18 Oct 2022 15:57:20 GMT
• Title: Number-Adaptive Prototype Learning for 3D Point Cloud Semantic Segmentation
• Authors: Yangheng Zhao, Jun Wang, Xiaolong Li, Yue Hu, Ce Zhang, Yanfeng Wang, and Siheng Chen
• Abstract summary: We propose to use an adaptive number of prototypes to dynamically describe the different point patterns within a semantic class. Our method achieves 2.3% mIoU improvement over the baseline model based on the point-wise classification paradigm.
• Score: 46.610620464184926
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D point cloud semantic segmentation is one of the fundamental tasks for 3D scene understanding and has been widely used in the metaverse applications. Many recent 3D semantic segmentation methods learn a single prototype (classifier weights) for each semantic class, and classify 3D points according to their nearest prototype. However, learning only one prototype for each class limits the model's ability to describe the high variance patterns within a class. Instead of learning a single prototype for each class, in this paper, we propose to use an adaptive number of prototypes to dynamically describe the different point patterns within a semantic class. With the powerful capability of vision transformer, we design a Number-Adaptive Prototype Learning (NAPL) model for point cloud semantic segmentation. To train our NAPL model, we propose a simple yet effective prototype dropout training strategy, which enables our model to adaptively produce prototypes for each class. The experimental results on SemanticKITTI dataset demonstrate that our method achieves 2.3% mIoU improvement over the baseline model based on the point-wise classification paradigm.

Related papers

• Rethinking Few-shot 3D Point Cloud Semantic Segmentation [62.80639841429669]
  This paper revisits few-shot 3D point cloud semantic segmentation (FS-PCS) We focus on two significant issues in the state-of-the-art: foreground leakage and sparse point distribution. To address these issues, we introduce a standardized FS-PCS setting, upon which a new benchmark is built.
  arXiv  Detail & Related papers  (2024-03-01T15:14:47Z)
• Towards Category Unification of 3D Single Object Tracking on Point Clouds [10.64650098374183]
  Category-specific models are provenly valuable methods in 3D single object tracking (SOT) regardless of Siamese or motion-centric paradigms. This paper first introduces unified models that can simultaneously track objects across all categories using a single network with shared model parameters.
  arXiv  Detail & Related papers  (2024-01-20T10:38:28Z)
• Prototype Adaption and Projection for Few- and Zero-shot 3D Point Cloud Semantic Segmentation [30.18333233940194]
  We address the challenging task of few-shot and zero-shot 3D point cloud semantic segmentation. Our proposed method surpasses state-of-the-art algorithms by a considerable 7.90% and 14.82% under the 2-way 1-shot setting on S3DIS and ScanNet benchmarks, respectively.
  arXiv  Detail & Related papers  (2023-05-23T17:58:05Z)
• Weakly Supervised 3D Point Cloud Segmentation via Multi-Prototype Learning [37.76664203157892]
  A fundamental challenge here lies in the large intra-class variations of local geometric structure, resulting in subclasses within a semantic class. We leverage this intuition and opt for maintaining an individual classifier for each subclass. Our hypothesis is also verified given the consistent discovery of semantic subclasses at no cost of additional annotations.
  arXiv  Detail & Related papers  (2022-05-06T11:07:36Z)
• Rethinking Semantic Segmentation: A Prototype View [126.59244185849838]
  We present a nonparametric semantic segmentation model based on non-learnable prototypes. Our framework yields compelling results over several datasets. We expect this work will provoke a rethink of the current de facto semantic segmentation model design.
  arXiv  Detail & Related papers  (2022-03-28T21:15:32Z)
• Dual Prototypical Contrastive Learning for Few-shot Semantic Segmentation [55.339405417090084]
  We propose a dual prototypical contrastive learning approach tailored to the few-shot semantic segmentation (FSS) task. The main idea is to encourage the prototypes more discriminative by increasing inter-class distance while reducing intra-class distance in prototype feature space. We demonstrate that the proposed dual contrastive learning approach outperforms state-of-the-art FSS methods on PASCAL-5i and COCO-20i datasets.
  arXiv  Detail & Related papers  (2021-11-09T08:14:50Z)
• Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer [112.95747173442754]
  A few-shot semantic segmentation model is typically composed of a CNN encoder, a CNN decoder and a simple classifier. Most existing methods meta-learn all three model components for fast adaptation to a new class. In this work we propose to simplify the meta-learning task by focusing solely on the simplest component, the classifier.
  arXiv  Detail & Related papers  (2021-08-06T10:20:08Z)
• Learn to Learn Metric Space for Few-Shot Segmentation of 3D Shapes [17.217954254022573]
  We introduce a meta-learning-based method for few-shot 3D shape segmentation where only a few labeled samples are provided for the unseen classes. We demonstrate the superior performance of our proposed on the ShapeNet part dataset under the few-shot scenario, compared with well-established baseline and state-of-the-art semi-supervised methods.
  arXiv  Detail & Related papers  (2021-07-07T01:47:00Z)
• Learning Feature Aggregation for Deep 3D Morphable Models [57.1266963015401]
  We propose an attention based module to learn mapping matrices for better feature aggregation across hierarchical levels. Our experiments show that through the end-to-end training of the mapping matrices, we achieve state-of-the-art results on a variety of 3D shape datasets.
  arXiv  Detail & Related papers  (2021-05-05T16:41:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.