Contrastive Predictive Autoencoders for Dynamic Point Cloud Self-Supervised Learning

• URL: http://arxiv.org/abs/2305.12959v1
• Date: Mon, 22 May 2023 12:09:51 GMT
• Title: Contrastive Predictive Autoencoders for Dynamic Point Cloud Self-Supervised Learning
• Authors: Xiaoxiao Sheng, Zhiqiang Shen, Gang Xiao
• Abstract summary: We design point cloud sequence based Contrastive Prediction and Reconstruction (CPR), to collaboratively learn more comprehensive representations. We conduct experiments on four point cloud sequence benchmarks, and report the results under multiple experimental settings.
• Score: 26.773995001469505
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a new self-supervised paradigm on point cloud sequence understanding. Inspired by the discriminative and generative self-supervised methods, we design two tasks, namely point cloud sequence based Contrastive Prediction and Reconstruction (CPR), to collaboratively learn more comprehensive spatiotemporal representations. Specifically, dense point cloud segments are first input into an encoder to extract embeddings. All but the last ones are then aggregated by a context-aware autoregressor to make predictions for the last target segment. Towards the goal of modeling multi-granularity structures, local and global contrastive learning are performed between predictions and targets. To further improve the generalization of representations, the predictions are also utilized to reconstruct raw point cloud sequences by a decoder, where point cloud colorization is employed to discriminate against different frames. By combining classic contrast and reconstruction paradigms, it makes the learned representations with both global discrimination and local perception. We conduct experiments on four point cloud sequence benchmarks, and report the results on action recognition and gesture recognition under multiple experimental settings. The performances are comparable with supervised methods and show powerful transferability.

Related papers

• Point Contrastive Prediction with Semantic Clustering for Self-Supervised Learning on Point Cloud Videos [71.20376514273367]
  We propose a unified point cloud video self-supervised learning framework for object-centric and scene-centric data. Our method outperforms supervised counterparts on a wide range of downstream tasks.
  arXiv  Detail & Related papers  (2023-08-18T02:17:47Z)
• GeoMAE: Masked Geometric Target Prediction for Self-supervised Point Cloud Pre-Training [16.825524577372473]
  We introduce a point cloud representation learning framework, based on geometric feature reconstruction. We identify three self-supervised learning objectives to peculiar point clouds, namely centroid prediction, normal estimation, and curvature prediction. Our pipeline is conceptually simple and it consists of two major steps: first, it randomly masks out groups of points, followed by a Transformer-based point cloud encoder.
  arXiv  Detail & Related papers  (2023-05-15T17:14:55Z)
• PointCMP: Contrastive Mask Prediction for Self-supervised Learning on Point Cloud Videos [58.18707835387484]
  We propose a contrastive mask prediction framework for self-supervised learning on point cloud videos. PointCMP employs a two-branch structure to achieve simultaneous learning of both local and globaltemporal information. Our framework achieves the state-of-the-art performance on benchmark datasets and outperforms existing full-supervised counterparts.
  arXiv  Detail & Related papers  (2023-05-06T15:47:48Z)
• Vector Quantized Wasserstein Auto-Encoder [57.29764749855623]
  We study learning deep discrete representations from the generative viewpoint. We endow discrete distributions over sequences of codewords and learn a deterministic decoder that transports the distribution over the sequences of codewords to the data distribution. We develop further theories to connect it with the clustering viewpoint of WS distance, allowing us to have a better and more controllable clustering solution.
  arXiv  Detail & Related papers  (2023-02-12T13:51:36Z)
• PointCAT: Contrastive Adversarial Training for Robust Point Cloud Recognition [111.55944556661626]
  We propose Point-Cloud Contrastive Adversarial Training (PointCAT) to boost the robustness of point cloud recognition models. We leverage a supervised contrastive loss to facilitate the alignment and uniformity of the hypersphere features extracted by the recognition model. To provide the more challenging corrupted point clouds, we adversarially train a noise generator along with the recognition model from the scratch.
  arXiv  Detail & Related papers  (2022-09-16T08:33:04Z)
• Self-Supervised Arbitrary-Scale Point Clouds Upsampling via Implicit Neural Representation [79.60988242843437]
  We propose a novel approach that achieves self-supervised and magnification-flexible point clouds upsampling simultaneously. Experimental results demonstrate that our self-supervised learning based scheme achieves competitive or even better performance than supervised learning based state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-04-18T07:18:25Z)
• Upsampling Autoencoder for Self-Supervised Point Cloud Learning [11.19408173558718]
  We propose a self-supervised pretraining model for point cloud learning without human annotations. Upsampling operation encourages the network to capture both high-level semantic information and low-level geometric information of the point cloud. We find that our UAE outperforms previous state-of-the-art methods in shape classification, part segmentation and point cloud upsampling tasks.
  arXiv  Detail & Related papers  (2022-03-21T07:20:37Z)
• Unsupervised Representation Learning for 3D Point Cloud Data [66.92077180228634]
  We propose a simple yet effective approach for unsupervised point cloud learning. In particular, we identify a very useful transformation which generates a good contrastive version of an original point cloud. We conduct experiments on three downstream tasks which are 3D object classification, shape part segmentation and scene segmentation.
  arXiv  Detail & Related papers  (2021-10-13T10:52:45Z)
• Self-Contrastive Learning with Hard Negative Sampling for Self-supervised Point Cloud Learning [17.55440737986014]
  We propose a novel self-contrastive learning for self-supervised point cloud representation learning. We exploit self-similar point cloud patches within a single point cloud as positive samples and otherwise negative ones to facilitate the task of contrastive learning. Experimental results show that the proposed method achieves state-of-the-art performance on widely used benchmark datasets.
  arXiv  Detail & Related papers  (2021-07-05T09:17:45Z)
• Self-Supervised Few-Shot Learning on Point Clouds [18.528929583956725]
  Recently, deep neural networks operating on labeled point clouds have shown promising results on supervised learning tasks like classification and segmentation. We propose two novel self-supervised pre-training tasks that encode a hierarchical partitioning of the point clouds. We show that supervised methods pre-trained with our self-supervised learning method significantly improve the accuracy of state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-09-29T17:32:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.