FreePoint: Unsupervised Point Cloud Instance Segmentation

• URL: http://arxiv.org/abs/2305.06973v2
• Date: Sat, 15 Jun 2024 21:29:54 GMT
• Title: FreePoint: Unsupervised Point Cloud Instance Segmentation
• Authors: Zhikai Zhang, Jian Ding, Li Jiang, Dengxin Dai, Gui-Song Xia,
• Abstract summary: We propose FreePoint, for underexplored unsupervised class-agnostic instance segmentation on point clouds. We represent point features by combining coordinates, colors, and self-supervised deep features. Based on the point features, we segment point clouds into coarse instance masks as pseudo labels, which are used to train a point cloud instance segmentation model.
• Score: 72.64540130803687
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Instance segmentation of point clouds is a crucial task in 3D field with numerous applications that involve localizing and segmenting objects in a scene. However, achieving satisfactory results requires a large number of manual annotations, which is a time-consuming and expensive process. To alleviate dependency on annotations, we propose a novel framework, FreePoint, for underexplored unsupervised class-agnostic instance segmentation on point clouds. In detail, we represent the point features by combining coordinates, colors, and self-supervised deep features. Based on the point features, we perform a bottom-up multicut algorithm to segment point clouds into coarse instance masks as pseudo labels, which are used to train a point cloud instance segmentation model. We propose an id-as-feature strategy at this stage to alleviate the randomness of the multicut algorithm and improve the pseudo labels' quality. During training, we propose a weakly-supervised two-step training strategy and corresponding losses to overcome the inaccuracy of coarse masks. FreePoint has achieved breakthroughs in unsupervised class-agnostic instance segmentation on point clouds and outperformed previous traditional methods by over 18.2% and a competitive concurrent work UnScene3D by 5.5% in AP. Additionally, when used as a pretext task and fine-tuned on S3DIS, FreePoint performs significantly better than existing self-supervised pre-training methods with limited annotations and surpasses CSC by 6.0% in AP with 10% annotation masks.

Related papers

• Clustering based Point Cloud Representation Learning for 3D Analysis [80.88995099442374]
  We propose a clustering based supervised learning scheme for point cloud analysis. Unlike current de-facto, scene-wise training paradigm, our algorithm conducts within-class clustering on the point embedding space. Our algorithm shows notable improvements on famous point cloud segmentation datasets.
  arXiv  Detail & Related papers  (2023-07-27T03:42:12Z)
• CPCM: Contextual Point Cloud Modeling for Weakly-supervised Point Cloud Semantic Segmentation [60.0893353960514]
  We study the task of weakly-supervised point cloud semantic segmentation with sparse annotations. We propose a Contextual Point Cloud Modeling ( CPCM) method that consists of two parts: a region-wise masking (RegionMask) strategy and a contextual masked training (CMT) method.
  arXiv  Detail & Related papers  (2023-07-19T04:41:18Z)
• Few-Shot 3D Point Cloud Semantic Segmentation via Stratified Class-Specific Attention Based Transformer Network [22.9434434107516]
  We develop a new multi-layer transformer network for few-shot point cloud semantic segmentation. Our method achieves the new state-of-the-art performance, with 15% less inference time, over existing few-shot 3D point cloud segmentation models.
  arXiv  Detail & Related papers  (2023-03-28T00:27:54Z)
• Weakly Supervised Semantic Segmentation for Large-Scale Point Cloud [69.36717778451667]
  Existing methods for large-scale point cloud semantic segmentation require expensive, tedious and error-prone manual point-wise annotations. We propose an effective weakly supervised method containing two components to solve the problem. The experimental results show the large gain against existing weakly supervised and comparable results to fully supervised methods.
  arXiv  Detail & Related papers  (2022-12-09T09:42:26Z)
• Learning Inter-Superpoint Affinity for Weakly Supervised 3D Instance Segmentation [10.968271388503986]
  We propose a 3D instance segmentation framework that can achieve good performance by annotating only one point for each instance. Our method achieves state-of-the-art performance in the weakly supervised point cloud instance segmentation task, and even outperforms some fully supervised methods.
  arXiv  Detail & Related papers  (2022-10-11T15:22:22Z)
• SCSS-Net: Superpoint Constrained Semi-supervised Segmentation Network for 3D Indoor Scenes [6.3364439467281315]
  We propose a superpoint constrained semi-supervised segmentation network for 3D point clouds, named as SCSS-Net. Specifically, we use the pseudo labels predicted from unlabeled point clouds for self-training, and the superpoints produced by geometry-based and color-based Region Growing algorithms are combined to modify and delete pseudo labels with low confidence.
  arXiv  Detail & Related papers  (2021-07-08T04:43:21Z)
• LRGNet: Learnable Region Growing for Class-Agnostic Point Cloud Segmentation [19.915593390338337]
  This research proposes a learnable region growing method for class-agnostic point cloud segmentation. The proposed method is able to segment any class of objects using a single deep neural network without any assumptions about their shapes and sizes.
  arXiv  Detail & Related papers  (2021-03-16T15:58:01Z)
• Few-shot 3D Point Cloud Semantic Segmentation [138.80825169240302]
  We propose a novel attention-aware multi-prototype transductive few-shot point cloud semantic segmentation method. Our proposed method shows significant and consistent improvements compared to baselines in different few-shot point cloud semantic segmentation settings.
  arXiv  Detail & Related papers  (2020-06-22T08:05:25Z)
• Weakly Supervised Semantic Point Cloud Segmentation:Towards 10X Fewer Labels [77.65554439859967]
  We propose a weakly supervised point cloud segmentation approach which requires only a tiny fraction of points to be labelled in the training stage. Experiments are done on three public datasets with different degrees of weak supervision.
  arXiv  Detail & Related papers  (2020-04-08T16:14:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.