Learning Inter-Superpoint Affinity for Weakly Supervised 3D Instance Segmentation

• URL: http://arxiv.org/abs/2210.05534v1
• Date: Tue, 11 Oct 2022 15:22:22 GMT
• Title: Learning Inter-Superpoint Affinity for Weakly Supervised 3D Instance Segmentation
• Authors: Linghua Tang and Le Hui and Jin Xie
• Abstract summary: We propose a 3D instance segmentation framework that can achieve good performance by annotating only one point for each instance. Our method achieves state-of-the-art performance in the weakly supervised point cloud instance segmentation task, and even outperforms some fully supervised methods.
• Score: 10.968271388503986
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Due to the few annotated labels of 3D point clouds, how to learn discriminative features of point clouds to segment object instances is a challenging problem. In this paper, we propose a simple yet effective 3D instance segmentation framework that can achieve good performance by annotating only one point for each instance. Specifically, to tackle extremely few labels for instance segmentation, we first oversegment the point cloud into superpoints in an unsupervised manner and extend the point-level annotations to the superpoint level. Then, based on the superpoint graph, we propose an inter-superpoint affinity mining module that considers the semantic and spatial relations to adaptively learn inter-superpoint affinity to generate high-quality pseudo labels via semantic-aware random walk. Finally, we propose a volume-aware instance refinement module to segment high-quality instances by applying volume constraints of objects in clustering on the superpoint graph. Extensive experiments on the ScanNet-v2 and S3DIS datasets demonstrate that our method achieves state-of-the-art performance in the weakly supervised point cloud instance segmentation task, and even outperforms some fully supervised methods.

Related papers

• Weakly Supervised 3D Instance Segmentation without Instance-level Annotations [57.615325809883636]
  3D semantic scene understanding tasks have achieved great success with the emergence of deep learning, but often require a huge amount of manually annotated training data. We propose the first weakly-supervised 3D instance segmentation method that only requires categorical semantic labels as supervision. By generating pseudo instance labels from categorical semantic labels, our designed approach can also assist existing methods for learning 3D instance segmentation at reduced annotation cost.
  arXiv  Detail & Related papers  (2023-08-03T12:30:52Z)
• GrowSP: Unsupervised Semantic Segmentation of 3D Point Clouds [12.597717463575478]
  We propose the first purely unsupervised method, called GrowSP, to successfully identify complex semantic classes for every point in 3D scenes. Our method consists of three major components, 1) the feature extractor to learn per-point features from input point clouds, 2) the superpoint constructor to progressively grow the sizes of superpoints, and 3) the semantic primitive clustering module to group superpoints into semantic elements. We extensively evaluate our method on multiple datasets, demonstrating superior performance over all unsupervised baselines and approaching the classic fully-supervised PointNet.
  arXiv  Detail & Related papers  (2023-05-25T18:11:21Z)
• FreePoint: Unsupervised Point Cloud Instance Segmentation [72.64540130803687]
  We propose FreePoint, for underexplored unsupervised class-agnostic instance segmentation on point clouds. We represent point features by combining coordinates, colors, and self-supervised deep features. Based on the point features, we segment point clouds into coarse instance masks as pseudo labels, which are used to train a point cloud instance segmentation model.
  arXiv  Detail & Related papers  (2023-05-11T16:56:26Z)
• Collaborative Propagation on Multiple Instance Graphs for 3D Instance Segmentation with Single-point Supervision [63.429704654271475]
  We propose a novel weakly supervised method RWSeg that only requires labeling one object with one point. With these sparse weak labels, we introduce a unified framework with two branches to propagate semantic and instance information. Specifically, we propose a Cross-graph Competing Random Walks (CRW) algorithm that encourages competition among different instance graphs.
  arXiv  Detail & Related papers  (2022-08-10T02:14:39Z)
• Box2Seg: Learning Semantics of 3D Point Clouds with Box-Level Supervision [65.19589997822155]
  We introduce a neural architecture, termed Box2Seg, to learn point-level semantics of 3D point clouds with bounding box-level supervision. We show that the proposed network can be trained with cheap, or even off-the-shelf bounding box-level annotations and subcloud-level tags.
  arXiv  Detail & Related papers  (2022-01-09T09:07:48Z)
• Dense Supervision Propagation for Weakly Supervised Semantic Segmentation on 3D Point Clouds [59.63231842439687]
  We train a semantic point cloud segmentation network with only a small portion of points being labeled. We propose a cross-sample feature reallocating module to transfer similar features and therefore re-route the gradients across two samples. Our weakly supervised method with only 10% and 1% of labels can produce compatible results with the fully supervised counterpart.
  arXiv  Detail & Related papers  (2021-07-23T14:34:57Z)
• SCSS-Net: Superpoint Constrained Semi-supervised Segmentation Network for 3D Indoor Scenes [6.3364439467281315]
  We propose a superpoint constrained semi-supervised segmentation network for 3D point clouds, named as SCSS-Net. Specifically, we use the pseudo labels predicted from unlabeled point clouds for self-training, and the superpoints produced by geometry-based and color-based Region Growing algorithms are combined to modify and delete pseudo labels with low confidence.
  arXiv  Detail & Related papers  (2021-07-08T04:43:21Z)
• SSPC-Net: Semi-supervised Semantic 3D Point Cloud Segmentation Network [21.818744369503197]
  We propose a semi-supervised semantic point cloud segmentation network, named SSPC-Net. We train the semantic segmentation network by inferring the labels of unlabeled points from the few annotated 3D points. Our method can achieve better performance than the current semi-supervised segmentation method with fewer annotated 3D points.
  arXiv  Detail & Related papers  (2021-04-16T02:37:27Z)
• Few-shot 3D Point Cloud Semantic Segmentation [138.80825169240302]
  We propose a novel attention-aware multi-prototype transductive few-shot point cloud semantic segmentation method. Our proposed method shows significant and consistent improvements compared to baselines in different few-shot point cloud semantic segmentation settings.
  arXiv  Detail & Related papers  (2020-06-22T08:05:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.