PointSmile: Point Self-supervised Learning via Curriculum Mutual Information

• URL: http://arxiv.org/abs/2301.12744v1
• Date: Mon, 30 Jan 2023 09:18:54 GMT
• Title: PointSmile: Point Self-supervised Learning via Curriculum Mutual Information
• Authors: Xin Li, Mingqiang Wei, Songcan Chen
• Abstract summary: We propose a reconstruction-free self-supervised learning paradigm by maximizing curriculum mutual information (CMI) across replicas of point cloud objects. PointSmile is designed to imitate human curriculum learning, starting with an easy curriculum and gradually increasing the difficulty of that curriculum. We demonstrate the effectiveness and robustness of PointSmile in downstream tasks including object classification and segmentation.
• Score: 33.74200235365997
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-supervised learning is attracting wide attention in point cloud processing. However, it is still not well-solved to gain discriminative and transferable features of point clouds for efficient training on downstream tasks, due to their natural sparsity and irregularity. We propose PointSmile, a reconstruction-free self-supervised learning paradigm by maximizing curriculum mutual information (CMI) across the replicas of point cloud objects. From the perspective of how-and-what-to-learn, PointSmile is designed to imitate human curriculum learning, i.e., starting with an easy curriculum and gradually increasing the difficulty of that curriculum. To solve "how-to-learn", we introduce curriculum data augmentation (CDA) of point clouds. CDA encourages PointSmile to learn from easy samples to hard ones, such that the latent space can be dynamically affected to create better embeddings. To solve "what-to-learn", we propose to maximize both feature- and class-wise CMI, for better extracting discriminative features of point clouds. Unlike most of existing methods, PointSmile does not require a pretext task, nor does it require cross-modal data to yield rich latent representations. We demonstrate the effectiveness and robustness of PointSmile in downstream tasks including object classification and segmentation. Extensive results show that our PointSmile outperforms existing self-supervised methods, and compares favorably with popular fully-supervised methods on various standard architectures.

Related papers

• PointMoment:Mixed-Moment-based Self-Supervised Representation Learning for 3D Point Clouds [11.980787751027872]
  We propose PointMoment, a novel framework for point cloud self-supervised representation learning. Our framework does not require any special techniques such as asymmetric network architectures, gradient stopping, etc.
  arXiv  Detail & Related papers  (2023-12-06T08:49:55Z)
• PointJEM: Self-supervised Point Cloud Understanding for Reducing Feature Redundancy via Joint Entropy Maximization [10.53900407467811]
  We propose PointJEM, a self-supervised representation learning method applied to the point cloud field. To reduce redundant information in the features, PointJEM maximizes the joint entropy between the different parts. PointJEM achieves competitive performance in downstream tasks such as classification and segmentation.
  arXiv  Detail & Related papers  (2023-12-06T08:21:42Z)
• Point Contrastive Prediction with Semantic Clustering for Self-Supervised Learning on Point Cloud Videos [71.20376514273367]
  We propose a unified point cloud video self-supervised learning framework for object-centric and scene-centric data. Our method outperforms supervised counterparts on a wide range of downstream tasks.
  arXiv  Detail & Related papers  (2023-08-18T02:17:47Z)
• Clustering based Point Cloud Representation Learning for 3D Analysis [80.88995099442374]
  We propose a clustering based supervised learning scheme for point cloud analysis. Unlike current de-facto, scene-wise training paradigm, our algorithm conducts within-class clustering on the point embedding space. Our algorithm shows notable improvements on famous point cloud segmentation datasets.
  arXiv  Detail & Related papers  (2023-07-27T03:42:12Z)
• A Survey of Label-Efficient Deep Learning for 3D Point Clouds [109.07889215814589]
  This paper presents the first comprehensive survey of label-efficient learning of point clouds. We propose a taxonomy that organizes label-efficient learning methods based on the data prerequisites provided by different types of labels. For each approach, we outline the problem setup and provide an extensive literature review that showcases relevant progress and challenges.
  arXiv  Detail & Related papers  (2023-05-31T12:54:51Z)
• Few-Shot Point Cloud Semantic Segmentation via Contrastive Self-Supervision and Multi-Resolution Attention [6.350163959194903]
  We propose a contrastive self-supervision framework for few-shot learning pretrain. Specifically, we implement a novel contrastive learning approach with a learnable augmentor for a 3D point cloud. We develop a multi-resolution attention module using both the nearest and farthest points to extract the local and global point information more effectively.
  arXiv  Detail & Related papers  (2023-02-21T07:59:31Z)
• PointCaM: Cut-and-Mix for Open-Set Point Cloud Learning [72.07350827773442]
  We propose to solve open-set point cloud learning using a novel Point Cut-and-Mix mechanism. We use the Unknown-Point Simulator to simulate out-of-distribution data in the training stage. The Unknown-Point Estimator module learns to exploit the point cloud's feature context for discriminating the known and unknown data.
  arXiv  Detail & Related papers  (2022-12-05T03:53:51Z)
• Data Augmentation-free Unsupervised Learning for 3D Point Cloud Understanding [61.30276576646909]
  We propose an augmentation-free unsupervised approach for point clouds to learn transferable point-level features via soft clustering, named SoftClu. We exploit the affiliation of points to their clusters as a proxy to enable self-training through a pseudo-label prediction task.
  arXiv  Detail & Related papers  (2022-10-06T10:18:16Z)
• Point Discriminative Learning for Unsupervised Representation Learning on 3D Point Clouds [54.31515001741987]
  We propose a point discriminative learning method for unsupervised representation learning on 3D point clouds. We achieve this by imposing a novel point discrimination loss on the middle level and global level point features. Our method learns powerful representations and achieves new state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-08-04T15:11:48Z)
• Self-Contrastive Learning with Hard Negative Sampling for Self-supervised Point Cloud Learning [17.55440737986014]
  We propose a novel self-contrastive learning for self-supervised point cloud representation learning. We exploit self-similar point cloud patches within a single point cloud as positive samples and otherwise negative ones to facilitate the task of contrastive learning. Experimental results show that the proposed method achieves state-of-the-art performance on widely used benchmark datasets.
  arXiv  Detail & Related papers  (2021-07-05T09:17:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.