AdaPoinTr: Diverse Point Cloud Completion with Adaptive Geometry-Aware Transformers

• URL: http://arxiv.org/abs/2301.04545v1
• Date: Wed, 11 Jan 2023 16:14:12 GMT
• Title: AdaPoinTr: Diverse Point Cloud Completion with Adaptive Geometry-Aware Transformers
• Authors: Xumin Yu, Yongming Rao, Ziyi Wang, Jiwen Lu, Jie Zhou
• Abstract summary: We present a new method that reformulates point cloud completion as a set-to-set translation problem. We design a new model, called PoinTr, which adopts a Transformer encoder-decoder architecture for point cloud completion. Our method attains 6.53 CD on PCN, 0.81 CD on ShapeNet-55 and 0.392 MMD on real-world KITTI.
• Score: 94.11915008006483
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we present a new method that reformulates point cloud completion as a set-to-set translation problem and design a new model, called PoinTr, which adopts a Transformer encoder-decoder architecture for point cloud completion. By representing the point cloud as a set of unordered groups of points with position embeddings, we convert the input data to a sequence of point proxies and employ the Transformers for generation. To facilitate Transformers to better leverage the inductive bias about 3D geometric structures of point clouds, we further devise a geometry-aware block that models the local geometric relationships explicitly. The migration of Transformers enables our model to better learn structural knowledge and preserve detailed information for point cloud completion. Taking a step towards more complicated and diverse situations, we further propose AdaPoinTr by developing an adaptive query generation mechanism and designing a novel denoising task during completing a point cloud. Coupling these two techniques enables us to train the model efficiently and effectively: we reduce training time (by 15x or more) and improve completion performance (over 20%). We also show our method can be extended to the scene-level point cloud completion scenario by designing a new geometry-enhanced semantic scene completion framework. Extensive experiments on the existing and newly-proposed datasets demonstrate the effectiveness of our method, which attains 6.53 CD on PCN, 0.81 CD on ShapeNet-55 and 0.392 MMD on real-world KITTI, surpassing other work by a large margin and establishing new state-of-the-arts on various benchmarks. Most notably, AdaPoinTr can achieve such promising performance with higher throughputs and fewer FLOPs compared with the previous best methods in practice. The code and datasets are available at https://github.com/yuxumin/PoinTr

Related papers

• PointRWKV: Efficient RWKV-Like Model for Hierarchical Point Cloud Learning [56.14518823931901]
  We present PointRWKV, a model of linear complexity derived from the RWKV model in the NLP field. We first propose to explore the global processing capabilities within PointRWKV blocks using modified multi-headed matrix-valued states. To extract local geometric features simultaneously, we design a parallel branch to encode the point cloud efficiently in a fixed radius near-neighbors graph with a graph stabilizer.
  arXiv  Detail & Related papers  (2024-05-24T05:02:51Z)
• Adaptive Point Transformer [88.28498667506165]
  Adaptive Point Cloud Transformer (AdaPT) is a standard PT model augmented by an adaptive token selection mechanism. AdaPT dynamically reduces the number of tokens during inference, enabling efficient processing of large point clouds.
  arXiv  Detail & Related papers  (2024-01-26T13:24:45Z)
• Patch-Wise Point Cloud Generation: A Divide-and-Conquer Approach [83.05340155068721]
  We devise a new 3d point cloud generation framework using a divide-and-conquer approach. All patch generators are based on learnable priors, which aim to capture the information of geometry primitives. Experimental results on a variety of object categories from the most popular point cloud dataset, ShapeNet, show the effectiveness of the proposed patch-wise point cloud generation.
  arXiv  Detail & Related papers  (2023-07-22T11:10:39Z)
• CpT: Convolutional Point Transformer for 3D Point Cloud Processing [10.389972581905]
  We present CpT: Convolutional point Transformer - a novel deep learning architecture for dealing with the unstructured nature of 3D point cloud data. CpT is an improvement over existing attention-based Convolutions Neural Networks as well as previous 3D point cloud processing transformers. Our model can serve as an effective backbone for various point cloud processing tasks when compared to the existing state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-11-21T17:45:55Z)
• Graph-Guided Deformation for Point Cloud Completion [35.10606375236494]
  We propose a Graph-Guided Deformation Network, which respectively regards the input data and intermediate generation as controlling and supporting points. Our key insight is to simulate the least square Laplacian deformation process via mesh deformation methods, which brings adaptivity for modeling variation in geometry details. We are the first to refine the point cloud completion task by mimicing traditional graphics algorithms with GCN-guided deformation.
  arXiv  Detail & Related papers  (2021-11-11T12:55:26Z)
• PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers [81.71904691925428]
  We present a new method that reformulates point cloud completion as a set-to-set translation problem. We also design a new model, called PoinTr, that adopts a transformer encoder-decoder architecture for point cloud completion. Our method outperforms state-of-the-art methods by a large margin on both the new benchmarks and the existing ones.
  arXiv  Detail & Related papers  (2021-08-19T17:58:56Z)
• Planning with Learned Dynamic Model for Unsupervised Point Cloud Registration [25.096635750142227]
  We develop a latent dynamic model of point clouds, consisting of a transformation network and evaluation network. We employ the cross-entropy method (CEM) to iteratively update the planning policy by maximizing the rewards in the point cloud registration process. Experimental results on ModelNet40 and 7Scene benchmark datasets demonstrate that our method can yield good registration performance in an unsupervised manner.
  arXiv  Detail & Related papers  (2021-08-05T13:47:11Z)
• SoftPoolNet: Shape Descriptor for Point Cloud Completion and Classification [93.54286830844134]
  We propose a method for 3D object completion and classification based on point clouds. For the decoder stage, we propose regional convolutions, a novel operator aimed at maximizing the global activation entropy. We evaluate our approach on different 3D tasks such as object completion and classification, achieving state-of-the-art accuracy.
  arXiv  Detail & Related papers  (2020-08-17T14:32:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.