COARSE3D: Class-Prototypes for Contrastive Learning in Weakly-Supervised 3D Point Cloud Segmentation

• URL: http://arxiv.org/abs/2210.01784v1
• Date: Tue, 4 Oct 2022 17:54:53 GMT
• Title: COARSE3D: Class-Prototypes for Contrastive Learning in Weakly-Supervised 3D Point Cloud Segmentation
• Authors: Rong Li and Anh-Quan Cao and Raoul de Charette
• Abstract summary: COARSE3D is a novel architecture-agnostic contrastive learning strategy for 3D segmentation. We leverage a prototype memory bank capturing class-wise global dataset information efficiently into a small number of prototypes acting as keys. Experiments show we outperform baselines on three challenging real-world outdoor datasets, working with as low as 0.001% annotations.
• Score: 16.072116380353393
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Annotation of large-scale 3D data is notoriously cumbersome and costly. As an alternative, weakly-supervised learning alleviates such a need by reducing the annotation by several order of magnitudes. We propose COARSE3D, a novel architecture-agnostic contrastive learning strategy for 3D segmentation. Since contrastive learning requires rich and diverse examples as keys and anchors, we leverage a prototype memory bank capturing class-wise global dataset information efficiently into a small number of prototypes acting as keys. An entropy-driven sampling technique then allows us to select good pixels from predictions as anchors. Experiments on three projection-based backbones show we outperform baselines on three challenging real-world outdoor datasets, working with as low as 0.001% annotations.

Related papers

• Bayesian Self-Training for Semi-Supervised 3D Segmentation [59.544558398992386]
  3D segmentation is a core problem in computer vision. densely labeling 3D point clouds to employ fully-supervised training remains too labor intensive and expensive. Semi-supervised training provides a more practical alternative, where only a small set of labeled data is given, accompanied by a larger unlabeled set.
  arXiv  Detail & Related papers  (2024-09-12T14:54:31Z)
• 3D Open-Vocabulary Panoptic Segmentation with 2D-3D Vision-Language Distillation [40.49322398635262]
  We propose the first method to tackle 3D open-vocabulary panoptic segmentation. Our model takes advantage of the fusion between learnable LiDAR features and dense frozen vision CLIP features. We propose two novel loss functions: object-level distillation loss and voxel-level distillation loss.
  arXiv  Detail & Related papers  (2024-01-04T18:39:32Z)
• You Only Need One Thing One Click: Self-Training for Weakly Supervised 3D Scene Understanding [107.06117227661204]
  We propose One Thing One Click'', meaning that the annotator only needs to label one point per object. We iteratively conduct the training and label propagation, facilitated by a graph propagation module. Our model can be compatible to 3D instance segmentation equipped with a point-clustering strategy.
  arXiv  Detail & Related papers  (2023-03-26T13:57:00Z)
• PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Models [56.324516906160234]
  Generalizable 3D part segmentation is important but challenging in vision and robotics. This paper explores an alternative way for low-shot part segmentation of 3D point clouds by leveraging a pretrained image-language model, GLIP. We transfer the rich knowledge from 2D to 3D through GLIP-based part detection on point cloud rendering and a novel 2D-to-3D label lifting algorithm.
  arXiv  Detail & Related papers  (2022-12-03T06:59:01Z)
• Primitive3D: 3D Object Dataset Synthesis from Randomly Assembled Primitives [44.03149443379618]
  We propose a cost-effective method for automatically generating a large amount of 3D objects with annotations. These objects are auto-annotated with part labels originating from primitives. Considering the large overhead of learning on the generated dataset, we propose a dataset distillation strategy.
  arXiv  Detail & Related papers  (2022-05-25T10:07:07Z)
• Open-Set Semi-Supervised Learning for 3D Point Cloud Understanding [62.17020485045456]
  It is commonly assumed in semi-supervised learning (SSL) that the unlabeled data are drawn from the same distribution as that of the labeled ones. We propose to selectively utilize unlabeled data through sample weighting, so that only conducive unlabeled data would be prioritized.
  arXiv  Detail & Related papers  (2022-05-02T16:09:17Z)
• Learning Compositional Shape Priors for Few-Shot 3D Reconstruction [36.40776735291117]
  We show that complex encoder-decoder architectures exploit large amounts of per-category data. We propose three ways to learn a class-specific global shape prior, directly from data. Experiments on the popular ShapeNet dataset show that our method outperforms a zero-shot baseline by over 40%.
  arXiv  Detail & Related papers  (2021-06-11T14:55:49Z)
• PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding [107.02479689909164]
  In this work, we aim at facilitating research on 3D representation learning. We measure the effect of unsupervised pre-training on a large source set of 3D scenes.
  arXiv  Detail & Related papers  (2020-07-21T17:59:22Z)
• D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features [51.04841465193678]
  We leverage a 3D fully convolutional network for 3D point clouds. We propose a novel and practical learning mechanism that densely predicts both a detection score and a description feature for each 3D point. Our method achieves state-of-the-art results in both indoor and outdoor scenarios.
  arXiv  Detail & Related papers  (2020-03-06T12:51:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.