GeoFormer: Learning Point Cloud Completion with Tri-Plane Integrated Transformer

• URL: http://arxiv.org/abs/2408.06596v1
• Date: Tue, 13 Aug 2024 03:15:36 GMT
• Title: GeoFormer: Learning Point Cloud Completion with Tri-Plane Integrated Transformer
• Authors: Jinpeng Yu, Binbin Huang, Yuxuan Zhang, Huaxia Li, Xu Tang, Shenghua Gao,
• Abstract summary: Point cloud completion aims to recover accurate global geometry and preserve fine-grained local details from partial point clouds. Conventional methods typically predict unseen points directly from 3D point cloud coordinates or use self-projected multi-view depth maps. We introduce a GeoFormer that simultaneously enhances the global geometric structure of the points and improves the local details.
• Score: 41.26276375114911
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Point cloud completion aims to recover accurate global geometry and preserve fine-grained local details from partial point clouds. Conventional methods typically predict unseen points directly from 3D point cloud coordinates or use self-projected multi-view depth maps to ease this task. However, these gray-scale depth maps cannot reach multi-view consistency, consequently restricting the performance. In this paper, we introduce a GeoFormer that simultaneously enhances the global geometric structure of the points and improves the local details. Specifically, we design a CCM Feature Enhanced Point Generator to integrate image features from multi-view consistent canonical coordinate maps (CCMs) and align them with pure point features, thereby enhancing the global geometry feature. Additionally, we employ the Multi-scale Geometry-aware Upsampler module to progressively enhance local details. This is achieved through cross attention between the multi-scale features extracted from the partial input and the features derived from previously estimated points. Extensive experiments on the PCN, ShapeNet-55/34, and KITTI benchmarks demonstrate that our GeoFormer outperforms recent methods, achieving the state-of-the-art performance. Our code is available at \href{https://github.com/Jinpeng-Yu/GeoFormer}{https://github.com/Jinpeng-Yu/GeoFormer}.

Related papers

• Boosting Cross-Domain Point Classification via Distilling Relational Priors from 2D Transformers [59.0181939916084]
  Traditional 3D networks mainly focus on local geometric details and ignore the topological structure between local geometries. We propose a novel Priors Distillation (RPD) method to extract priors from the well-trained transformers on massive images. Experiments on the PointDA-10 and the Sim-to-Real datasets verify that the proposed method consistently achieves the state-of-the-art performance of UDA for point cloud classification.
  arXiv  Detail & Related papers  (2024-07-26T06:29:09Z)
• Geometrically-driven Aggregation for Zero-shot 3D Point Cloud Understanding [11.416392706435415]
  Zero-shot 3D point cloud understanding can be achieved via 2D Vision-Language Models (VLMs) Existing strategies directly map Vision-Language Models from 2D pixels of rendered or captured views to 3D points, overlooking the inherent and expressible point cloud geometric structure. We introduce the first training-free aggregation technique that leverages the point cloud's 3D geometric structure to improve the quality of the transferred Vision-Language Models.
  arXiv  Detail & Related papers  (2023-12-04T12:30:07Z)
• GeoDTR+: Toward generic cross-view geolocalization via geometric disentanglement [20.346145927174373]
  Cross-View Geo-Localization (CVGL) estimates the location of a ground image by matching it to a geo-tagged aerial image in a database. Existing methods still suffer from poor performance in cross-area evaluation, in which the training and testing data are captured from completely distinct areas. We attribute this deficiency to the lack of ability to extract the geometric layout of visual features and models' overfitting to low-level details. In this work, we propose GeoDTR+ with an enhanced GLE module that better models the correlations among visual features.
  arXiv  Detail & Related papers  (2023-08-18T15:32:01Z)
• GeoSpark: Sparking up Point Cloud Segmentation with Geometry Clue [25.747471104753426]
  GeoSpark is a Plug-in module that incorporates geometry clues into the network to Spark up feature learning and downsampling. For feature aggregation, GeoSpark improves by allowing the network to learn from both local points and neighboring geometry partitions. GeoSpark utilizes geometry partition information to guide the downsampling process, where points with unique features are preserved while redundant points are fused.
  arXiv  Detail & Related papers  (2023-03-14T23:30:46Z)
• Fusing Visual Appearance and Geometry for Multi-modality 6DoF Object Tracking [21.74515335906769]
  We develop a multi-modality tracker that fuses information from visual appearance and geometry to estimate object poses. The algorithm extends our previous method ICG, which uses geometry, to additionally consider surface appearance.
  arXiv  Detail & Related papers  (2023-02-22T15:53:00Z)
• Flattening-Net: Deep Regular 2D Representation for 3D Point Cloud Analysis [66.49788145564004]
  We present an unsupervised deep neural architecture called Flattening-Net to represent irregular 3D point clouds of arbitrary geometry and topology. Our methods perform favorably against the current state-of-the-art competitors.
  arXiv  Detail & Related papers  (2022-12-17T15:05:25Z)
• GeoSegNet: Point Cloud Semantic Segmentation via Geometric Encoder-Decoder Modeling [39.35429984469557]
  We present a robust semantic segmentation network dubbed GeoSegNet. Our GeoSegNet consists of a multi-geometry based encoder and a boundary-guided decoder. Experiments show obvious improvements of our method over its competitors in terms of the overall segmentation accuracy and object boundary clearness.
  arXiv  Detail & Related papers  (2022-07-14T09:24:05Z)
• Learning Geometry-Disentangled Representation for Complementary Understanding of 3D Object Point Cloud [50.56461318879761]
  We propose Geometry-Disentangled Attention Network (GDANet) for 3D image processing. GDANet disentangles point clouds into contour and flat part of 3D objects, respectively denoted by sharp and gentle variation components. Experiments on 3D object classification and segmentation benchmarks demonstrate that GDANet achieves the state-of-the-arts with fewer parameters.
  arXiv  Detail & Related papers  (2020-12-20T13:35:00Z)
• Cascaded Refinement Network for Point Cloud Completion [74.80746431691938]
  We propose a cascaded refinement network together with a coarse-to-fine strategy to synthesize the detailed object shapes. Considering the local details of partial input with the global shape information together, we can preserve the existing details in the incomplete point set. We also design a patch discriminator that guarantees every local area has the same pattern with the ground truth to learn the complicated point distribution.
  arXiv  Detail & Related papers  (2020-04-07T13:03:29Z)
• PUGeo-Net: A Geometry-centric Network for 3D Point Cloud Upsampling [103.09504572409449]
  We propose a novel deep neural network based method, called PUGeo-Net, to generate uniform dense point clouds. Thanks to its geometry-centric nature, PUGeo-Net works well for both CAD models with sharp features and scanned models with rich geometric details.
  arXiv  Detail & Related papers  (2020-02-24T14:13:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.