Learning Latent Part-Whole Hierarchies for Point Clouds

• URL: http://arxiv.org/abs/2211.07082v1
• Date: Mon, 14 Nov 2022 03:17:33 GMT
• Title: Learning Latent Part-Whole Hierarchies for Point Clouds
• Authors: Xiang Gao, Wei Hu, Renjie Liao
• Abstract summary: We propose an encoder-decoder style latent variable model that explicitly learns the part-whole hierarchies for the point cloud segmentation. The proposed method achieves state-of-the-art performance in not only top-level part segmentation but also middle-level latent subpart segmentation.
• Score: 41.288934432615676
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Strong evidence suggests that humans perceive the 3D world by parsing visual scenes and objects into part-whole hierarchies. Although deep neural networks have the capability of learning powerful multi-level representations, they can not explicitly model part-whole hierarchies, which limits their expressiveness and interpretability in processing 3D vision data such as point clouds. To this end, we propose an encoder-decoder style latent variable model that explicitly learns the part-whole hierarchies for the multi-level point cloud segmentation. Specifically, the encoder takes a point cloud as input and predicts the per-point latent subpart distribution at the middle level. The decoder takes the latent variable and the feature from the encoder as an input and predicts the per-point part distribution at the top level. During training, only annotated part labels at the top level are provided, thus making the whole framework weakly supervised. We explore two kinds of approximated inference algorithms, i.e., most-probable-latent and Monte Carlo methods, and three stochastic gradient estimations for learning discrete latent variables, i.e., straight-through, REINFORCE, and pathwise estimators. Experimental results on the PartNet dataset show that the proposed method achieves state-of-the-art performance in not only top-level part segmentation but also middle-level latent subpart segmentation.

Related papers

• GrowSP: Unsupervised Semantic Segmentation of 3D Point Clouds [12.597717463575478]
  We propose the first purely unsupervised method, called GrowSP, to successfully identify complex semantic classes for every point in 3D scenes. Our method consists of three major components, 1) the feature extractor to learn per-point features from input point clouds, 2) the superpoint constructor to progressively grow the sizes of superpoints, and 3) the semantic primitive clustering module to group superpoints into semantic elements. We extensively evaluate our method on multiple datasets, demonstrating superior performance over all unsupervised baselines and approaching the classic fully-supervised PointNet.
  arXiv  Detail & Related papers  (2023-05-25T18:11:21Z)
• Unsupervised Representation Learning for 3D Point Cloud Data [66.92077180228634]
  We propose a simple yet effective approach for unsupervised point cloud learning. In particular, we identify a very useful transformation which generates a good contrastive version of an original point cloud. We conduct experiments on three downstream tasks which are 3D object classification, shape part segmentation and scene segmentation.
  arXiv  Detail & Related papers  (2021-10-13T10:52:45Z)
• UPDesc: Unsupervised Point Descriptor Learning for Robust Registration [54.95201961399334]
  UPDesc is an unsupervised method to learn point descriptors for robust point cloud registration. We show that our learned descriptors yield superior performance over existing unsupervised methods.
  arXiv  Detail & Related papers  (2021-08-05T17:11:08Z)
• Point Discriminative Learning for Unsupervised Representation Learning on 3D Point Clouds [54.31515001741987]
  We propose a point discriminative learning method for unsupervised representation learning on 3D point clouds. We achieve this by imposing a novel point discrimination loss on the middle level and global level point features. Our method learns powerful representations and achieves new state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-08-04T15:11:48Z)
• Omni-supervised Point Cloud Segmentation via Gradual Receptive Field Component Reasoning [41.83979510282989]
  We bring the first omni-scale supervision method to point cloud segmentation via the proposed gradual Receptive Field Component Reasoning (RFCR) Our method brings new state-of-the-art performances for S3DIS as well as Semantic3D and ranks the 1st in the ScanNet benchmark among all the point-based methods.
  arXiv  Detail & Related papers  (2021-05-21T08:32:02Z)
• Unsupervised Feedforward Feature (UFF) Learning for Point Cloud Classification and Segmentation [57.62713515497585]
  Unsupervised feedforward feature learning is proposed for joint classification and segmentation of 3D point clouds. The UFF method exploits statistical correlations of points in a point cloud set to learn shape and point features in a one-pass feedforward manner. It learns global shape features through the encoder and local point features through the encoder-decoder architecture.
  arXiv  Detail & Related papers  (2020-09-02T18:25:25Z)
• Campus3D: A Photogrammetry Point Cloud Benchmark for Hierarchical Understanding of Outdoor Scene [76.4183572058063]
  We present a richly-annotated 3D point cloud dataset for multiple outdoor scene understanding tasks. The dataset has been point-wisely annotated with both hierarchical and instance-based labels. We formulate a hierarchical learning problem for 3D point cloud segmentation and propose a measurement evaluating consistency across various hierarchies.
  arXiv  Detail & Related papers  (2020-08-11T19:10:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.