Masked Discrimination for Self-Supervised Learning on Point Clouds
- URL: http://arxiv.org/abs/2203.11183v1
- Date: Mon, 21 Mar 2022 17:57:34 GMT
- Title: Masked Discrimination for Self-Supervised Learning on Point Clouds
- Authors: Haotian Liu, Mu Cai, Yong Jae Lee
- Abstract summary: Masked autoencoding has achieved great success for self-supervised learning in the image and language domains.
Standard backbones like PointNet are unable to properly handle the training versus testing distribution mismatch introduced by masking during training.
We bridge this gap by proposing a discriminative mask pretraining Transformer framework, MaskPoint, for point clouds.
- Score: 27.652157544218234
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Masked autoencoding has achieved great success for self-supervised learning
in the image and language domains. However, mask based pretraining has yet to
show benefits for point cloud understanding, likely due to standard backbones
like PointNet being unable to properly handle the training versus testing
distribution mismatch introduced by masking during training. In this paper, we
bridge this gap by proposing a discriminative mask pretraining Transformer
framework, MaskPoint}, for point clouds. Our key idea is to represent the point
cloud as discrete occupancy values (1 if part of the point cloud; 0 if not),
and perform simple binary classification between masked object points and
sampled noise points as the proxy task. In this way, our approach is robust to
the point sampling variance in point clouds, and facilitates learning rich
representations. We evaluate our pretrained models across several downstream
tasks, including 3D shape classification, segmentation, and real-word object
detection, and demonstrate state-of-the-art results while achieving a
significant pretraining speedup (e.g., 4.1x on ScanNet) compared to the prior
state-of-the-art Transformer baseline. Code will be publicly available at
https://github.com/haotian-liu/MaskPoint.
Related papers
- CPCM: Contextual Point Cloud Modeling for Weakly-supervised Point Cloud
Semantic Segmentation [60.0893353960514]
We study the task of weakly-supervised point cloud semantic segmentation with sparse annotations.
We propose a Contextual Point Cloud Modeling ( CPCM) method that consists of two parts: a region-wise masking (RegionMask) strategy and a contextual masked training (CMT) method.
arXiv Detail & Related papers (2023-07-19T04:41:18Z) - Self-supervised adversarial masking for 3D point cloud representation
learning [0.38233569758620056]
We introduce PointCAM, a novel adversarial method for learning a masking function for point clouds.
Compared to previous techniques, we postulate applying an auxiliary network that learns how to select masks instead of choosing them randomly.
Our results show that the learned masking function achieves state-of-the-art or competitive performance on various downstream tasks.
arXiv Detail & Related papers (2023-07-11T15:11:06Z) - FreePoint: Unsupervised Point Cloud Instance Segmentation [72.64540130803687]
We propose FreePoint, for underexplored unsupervised class-agnostic instance segmentation on point clouds.
We represent point features by combining coordinates, colors, and self-supervised deep features.
Based on the point features, we segment point clouds into coarse instance masks as pseudo labels, which are used to train a point cloud instance segmentation model.
arXiv Detail & Related papers (2023-05-11T16:56:26Z) - Weakly Supervised Semantic Segmentation for Large-Scale Point Cloud [69.36717778451667]
Existing methods for large-scale point cloud semantic segmentation require expensive, tedious and error-prone manual point-wise annotations.
We propose an effective weakly supervised method containing two components to solve the problem.
The experimental results show the large gain against existing weakly supervised and comparable results to fully supervised methods.
arXiv Detail & Related papers (2022-12-09T09:42:26Z) - Point-McBert: A Multi-choice Self-supervised Framework for Point Cloud
Pre-training [6.037383467521294]
We propose Point-McBert, a pre-training framework with eased and refined supervision signals.
Specifically, we ease the previous single-choice constraint on patches, and provide multi-choice token ids for each patch as supervision.
Our method achieves 94.1% accuracy on ModelNet40, 84.28% accuracy on the hardest setting of ScanObjectNN and new state-of-the-art performance on few-shot learning.
arXiv Detail & Related papers (2022-07-27T00:34:33Z) - Masked Surfel Prediction for Self-Supervised Point Cloud Learning [40.16043026141161]
We make the first attempt to consider the local geometry information explicitly into the masked auto-encoding, and propose a novel Masked Surfel Prediction (MaskSurf) method.
Specifically, given the input point cloud masked at a high ratio, we learn a transformer-based encoder-decoder network to estimate the underlying masked surfels.
MaskSurf is validated on six downstream tasks under three fine-tuning strategies.
arXiv Detail & Related papers (2022-07-07T06:47:26Z) - Masked Autoencoders in 3D Point Cloud Representation Learning [7.617783375837524]
We propose masked Autoencoders in 3D point cloud representation learning (abbreviated as MAE3D)
We first split the input point cloud into patches and mask a portion of them, then use our Patch Embedding Module to extract the features of unmasked patches.
Comprehensive experiments demonstrate that the local features extracted by our MAE3D from point cloud patches are beneficial for downstream classification tasks.
arXiv Detail & Related papers (2022-07-04T16:13:27Z) - What You See is What You Classify: Black Box Attributions [61.998683569022006]
We train a deep network, the Explainer, to predict attributions for a pre-trained black-box classifier, the Explanandum.
Unlike most existing approaches, ours is capable of directly generating very distinct class-specific masks.
We show that our attributions are superior to established methods both visually and quantitatively.
arXiv Detail & Related papers (2022-05-23T12:30:04Z) - Self-Supervised Arbitrary-Scale Point Clouds Upsampling via Implicit
Neural Representation [79.60988242843437]
We propose a novel approach that achieves self-supervised and magnification-flexible point clouds upsampling simultaneously.
Experimental results demonstrate that our self-supervised learning based scheme achieves competitive or even better performance than supervised learning based state-of-the-art methods.
arXiv Detail & Related papers (2022-04-18T07:18:25Z) - Unsupervised Representation Learning for 3D Point Cloud Data [66.92077180228634]
We propose a simple yet effective approach for unsupervised point cloud learning.
In particular, we identify a very useful transformation which generates a good contrastive version of an original point cloud.
We conduct experiments on three downstream tasks which are 3D object classification, shape part segmentation and scene segmentation.
arXiv Detail & Related papers (2021-10-13T10:52:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.