Generative PointNet: Deep Energy-Based Learning on Unordered Point Sets for 3D Generation, Reconstruction and Classification

• URL: http://arxiv.org/abs/2004.01301v2
• Date: Wed, 7 Apr 2021 14:37:08 GMT
• Title: Generative PointNet: Deep Energy-Based Learning on Unordered Point Sets for 3D Generation, Reconstruction and Classification
• Authors: Jianwen Xie, Yifei Xu, Zilong Zheng, Song-Chun Zhu, Ying Nian Wu
• Abstract summary: We propose a generative model of unordered point sets, such as point clouds, in the form of an energy-based model. We call our model the Generative PointNet because it can be derived from the discriminative PointNet.
• Score: 136.57669231704858
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a generative model of unordered point sets, such as point clouds, in the form of an energy-based model, where the energy function is parameterized by an input-permutation-invariant bottom-up neural network. The energy function learns a coordinate encoding of each point and then aggregates all individual point features into an energy for the whole point cloud. We call our model the Generative PointNet because it can be derived from the discriminative PointNet. Our model can be trained by MCMC-based maximum likelihood learning (as well as its variants), without the help of any assisting networks like those in GANs and VAEs. Unlike most point cloud generators that rely on hand-crafted distance metrics, our model does not require any hand-crafted distance metric for the point cloud generation, because it synthesizes point clouds by matching observed examples in terms of statistical properties defined by the energy function. Furthermore, we can learn a short-run MCMC toward the energy-based model as a flow-like generator for point cloud reconstruction and interpolation. The learned point cloud representation can be useful for point cloud classification. Experiments demonstrate the advantages of the proposed generative model of point clouds.

Related papers

• A Hybrid Generative and Discriminative PointNet on Unordered Point Sets [6.282930329443868]
  This paper proposes GDPNet, the first hybrid Generative and Discriminative PointNet. Our GDPNet retains strong discriminative power of modern PointNet classifiers, while generating point cloud samples rivaling state-of-the-art generative approaches.
  arXiv  Detail & Related papers  (2024-04-19T14:52:25Z)
• Point Cloud Pre-training with Diffusion Models [62.12279263217138]
  We propose a novel pre-training method called Point cloud Diffusion pre-training (PointDif) PointDif achieves substantial improvement across various real-world datasets for diverse downstream tasks such as classification, segmentation and detection.
  arXiv  Detail & Related papers  (2023-11-25T08:10:05Z)
• Patch-Wise Point Cloud Generation: A Divide-and-Conquer Approach [83.05340155068721]
  We devise a new 3d point cloud generation framework using a divide-and-conquer approach. All patch generators are based on learnable priors, which aim to capture the information of geometry primitives. Experimental results on a variety of object categories from the most popular point cloud dataset, ShapeNet, show the effectiveness of the proposed patch-wise point cloud generation.
  arXiv  Detail & Related papers  (2023-07-22T11:10:39Z)
• Variational Relational Point Completion Network for Robust 3D Classification [59.80993960827833]
  Vari point cloud completion methods tend to generate global shape skeletons hence lack fine local details. This paper proposes a variational framework, point Completion Network (VRCNet) with two appealing properties. VRCNet shows great generalizability and robustness on real-world point cloud scans.
  arXiv  Detail & Related papers  (2023-04-18T17:03:20Z)
• Controllable Mesh Generation Through Sparse Latent Point Diffusion Models [105.83595545314334]
  We design a novel sparse latent point diffusion model for mesh generation. Our key insight is to regard point clouds as an intermediate representation of meshes, and model the distribution of point clouds instead. Our proposed sparse latent point diffusion model achieves superior performance in terms of generation quality and controllability.
  arXiv  Detail & Related papers  (2023-03-14T14:25:29Z)
• Representing Point Clouds with Generative Conditional Invertible Flow Networks [15.280751949071016]
  We propose a simple yet effective method to represent point clouds as sets of samples drawn from a cloud-specific probability distribution. We show that our method leverages generative invertible flow networks to learn embeddings as well as to generate point clouds. Our model offers competitive or superior quantitative results on benchmark datasets.
  arXiv  Detail & Related papers  (2020-10-07T18:30:47Z)
• Multi-scale Receptive Fields Graph Attention Network for Point Cloud Classification [35.88116404702807]
  The proposed MRFGAT architecture is tested on ModelNet10 and ModelNet40 datasets. Results show it achieves state-of-the-art performance in shape classification tasks.
  arXiv  Detail & Related papers  (2020-09-28T13:01:28Z)
• Permutation Matters: Anisotropic Convolutional Layer for Learning on Point Clouds [145.79324955896845]
  We propose a permutable anisotropic convolutional operation (PAI-Conv) that calculates soft-permutation matrices for each point. Experiments on point clouds demonstrate that PAI-Conv produces competitive results in classification and semantic segmentation tasks.
  arXiv  Detail & Related papers  (2020-05-27T02:42:29Z)
• Airborne LiDAR Point Cloud Classification with Graph Attention Convolution Neural Network [5.69168146446103]
  We present a graph attention convolution neural network (GACNN) that can be directly applied to the classification of unstructured 3D point clouds obtained by airborne LiDAR. Based on the proposed graph attention convolution module, we further design an end-to-end encoder-decoder network, named GACNN, to capture multiscale features of the point clouds. Experiments on the ISPRS 3D labeling dataset show that the proposed model achieves a new state-of-the-art performance in terms of average F1 score (71.5%) and a satisfying overall accuracy (83.2%)
  arXiv  Detail & Related papers  (2020-04-20T05:12:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.