CPCM: Contextual Point Cloud Modeling for Weakly-supervised Point Cloud Semantic Segmentation

• URL: http://arxiv.org/abs/2307.10316v1
• Date: Wed, 19 Jul 2023 04:41:18 GMT
• Title: CPCM: Contextual Point Cloud Modeling for Weakly-supervised Point Cloud Semantic Segmentation
• Authors: Lizhao Liu, Zhuangwei Zhuang, Shangxin Huang, Xunlong Xiao, Tianhang Xiang, Cen Chen, Jingdong Wang and Mingkui Tan
• Abstract summary: We study the task of weakly-supervised point cloud semantic segmentation with sparse annotations. We propose a Contextual Point Cloud Modeling ( CPCM) method that consists of two parts: a region-wise masking (RegionMask) strategy and a contextual masked training (CMT) method.
• Score: 60.0893353960514
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study the task of weakly-supervised point cloud semantic segmentation with sparse annotations (e.g., less than 0.1% points are labeled), aiming to reduce the expensive cost of dense annotations. Unfortunately, with extremely sparse annotated points, it is very difficult to extract both contextual and object information for scene understanding such as semantic segmentation. Motivated by masked modeling (e.g., MAE) in image and video representation learning, we seek to endow the power of masked modeling to learn contextual information from sparsely-annotated points. However, directly applying MAE to 3D point clouds with sparse annotations may fail to work. First, it is nontrivial to effectively mask out the informative visual context from 3D point clouds. Second, how to fully exploit the sparse annotations for context modeling remains an open question. In this paper, we propose a simple yet effective Contextual Point Cloud Modeling (CPCM) method that consists of two parts: a region-wise masking (RegionMask) strategy and a contextual masked training (CMT) method. Specifically, RegionMask masks the point cloud continuously in geometric space to construct a meaningful masked prediction task for subsequent context learning. CMT disentangles the learning of supervised segmentation and unsupervised masked context prediction for effectively learning the very limited labeled points and mass unlabeled points, respectively. Extensive experiments on the widely-tested ScanNet V2 and S3DIS benchmarks demonstrate the superiority of CPCM over the state-of-the-art.

Related papers

• FreePoint: Unsupervised Point Cloud Instance Segmentation [72.64540130803687]
  We propose FreePoint, for underexplored unsupervised class-agnostic instance segmentation on point clouds. We represent point features by combining coordinates, colors, and self-supervised deep features. Based on the point features, we segment point clouds into coarse instance masks as pseudo labels, which are used to train a point cloud instance segmentation model.
  arXiv  Detail & Related papers  (2023-05-11T16:56:26Z)
• PointDC:Unsupervised Semantic Segmentation of 3D Point Clouds via Cross-modal Distillation and Super-Voxel Clustering [32.18716273358168]
  We take the first attempt for fully unsupervised semantic segmentation of point clouds. We propose a novel framework, PointDC, comprised of two steps that handle the aforementioned problems. PointDC yields a significant improvement over the prior state-of-the-art unsupervised methods.
  arXiv  Detail & Related papers  (2023-04-18T12:58:21Z)
• Point2Vec for Self-Supervised Representation Learning on Point Clouds [66.53955515020053]
  We extend data2vec to the point cloud domain and report encouraging results on several downstream tasks. We propose point2vec, which unleashes the full potential of data2vec-like pre-training on point clouds.
  arXiv  Detail & Related papers  (2023-03-29T10:08:29Z)
• Weakly Supervised Semantic Segmentation for Large-Scale Point Cloud [69.36717778451667]
  Existing methods for large-scale point cloud semantic segmentation require expensive, tedious and error-prone manual point-wise annotations. We propose an effective weakly supervised method containing two components to solve the problem. The experimental results show the large gain against existing weakly supervised and comparable results to fully supervised methods.
  arXiv  Detail & Related papers  (2022-12-09T09:42:26Z)
• Masked Discrimination for Self-Supervised Learning on Point Clouds [27.652157544218234]
  Masked autoencoding has achieved great success for self-supervised learning in the image and language domains. Standard backbones like PointNet are unable to properly handle the training versus testing distribution mismatch introduced by masking during training. We bridge this gap by proposing a discriminative mask pretraining Transformer framework, MaskPoint, for point clouds.
  arXiv  Detail & Related papers  (2022-03-21T17:57:34Z)
• Box2Seg: Learning Semantics of 3D Point Clouds with Box-Level Supervision [65.19589997822155]
  We introduce a neural architecture, termed Box2Seg, to learn point-level semantics of 3D point clouds with bounding box-level supervision. We show that the proposed network can be trained with cheap, or even off-the-shelf bounding box-level annotations and subcloud-level tags.
  arXiv  Detail & Related papers  (2022-01-09T09:07:48Z)
• Unsupervised Representation Learning for 3D Point Cloud Data [66.92077180228634]
  We propose a simple yet effective approach for unsupervised point cloud learning. In particular, we identify a very useful transformation which generates a good contrastive version of an original point cloud. We conduct experiments on three downstream tasks which are 3D object classification, shape part segmentation and scene segmentation.
  arXiv  Detail & Related papers  (2021-10-13T10:52:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.