Zero-Shot Point Cloud Registration

• URL: http://arxiv.org/abs/2312.03032v2
• Date: Fri, 8 Dec 2023 17:33:12 GMT
• Title: Zero-Shot Point Cloud Registration
• Authors: Weijie Wang, Guofeng Mei, Bin Ren, Xiaoshui Huang, Fabio Poiesi, Luc Van Gool, Nicu Sebe, Bruno Lepri
• Abstract summary: ZeroReg is the first zero-shot point cloud registration approach that eliminates the need for training on point cloud datasets. The cornerstone of ZeroReg is the novel transfer of image features from keypoints to the point cloud, enriched by aggregating information from 3D geometric neighborhoods. On benchmarks such as 3DMatch, 3DLoMatch, and ScanNet, ZeroReg achieves impressive Recall Ratios (RR) of over 84%, 46%, and 75%, respectively.
• Score: 94.39796531154303
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Learning-based point cloud registration approaches have significantly outperformed their traditional counterparts. However, they typically require extensive training on specific datasets. In this paper, we propose , the first zero-shot point cloud registration approach that eliminates the need for training on point cloud datasets. The cornerstone of ZeroReg is the novel transfer of image features from keypoints to the point cloud, enriched by aggregating information from 3D geometric neighborhoods. Specifically, we extract keypoints and features from 2D image pairs using a frozen pretrained 2D backbone. These features are then projected in 3D, and patches are constructed by searching for neighboring points. We integrate the geometric and visual features of each point using our novel parameter-free geometric decoder. Subsequently, the task of determining correspondences between point clouds is formulated as an optimal transport problem. Extensive evaluations of ZeroReg demonstrate its competitive performance against both traditional and learning-based methods. On benchmarks such as 3DMatch, 3DLoMatch, and ScanNet, ZeroReg achieves impressive Recall Ratios (RR) of over 84%, 46%, and 75%, respectively.

Related papers

• PointRegGPT: Boosting 3D Point Cloud Registration using Generative Point-Cloud Pairs for Training [90.06520673092702]
  We present PointRegGPT, boosting 3D point cloud registration using generative point-cloud pairs for training. To our knowledge, this is the first generative approach that explores realistic data generation for indoor point cloud registration.
  arXiv  Detail & Related papers  (2024-07-19T06:29:57Z)
• Zero-shot Point Cloud Completion Via 2D Priors [52.72867922938023]
  3D point cloud completion is designed to recover complete shapes from partially observed point clouds. We propose a zero-shot framework aimed at completing partially observed point clouds across any unseen categories.
  arXiv  Detail & Related papers  (2024-04-10T08:02:17Z)
• SphereNet: Learning a Noise-Robust and General Descriptor for Point Cloud Registration [5.641476052216014]
  We introduce SphereNet to learn a noise-robust and unseen-general descriptor for point cloud registration. A new benchmark 3DMatch-noise with strong noise is introduced to evaluate our methods. It sets a new state-of-the-art performance for the 3DMatch and 3DLoMatch benchmarks with 93.5% and 75.6% registration recall.
  arXiv  Detail & Related papers  (2023-07-18T15:37:35Z)
• Variational Relational Point Completion Network for Robust 3D Classification [59.80993960827833]
  Vari point cloud completion methods tend to generate global shape skeletons hence lack fine local details. This paper proposes a variational framework, point Completion Network (VRCNet) with two appealing properties. VRCNet shows great generalizability and robustness on real-world point cloud scans.
  arXiv  Detail & Related papers  (2023-04-18T17:03:20Z)
• ImLoveNet: Misaligned Image-supported Registration Network for Low-overlap Point Cloud Pairs [14.377604289952188]
  Low-overlap regions between paired point clouds make the captured features very low-confidence. We propose a misaligned image supported registration network for low-overlap point cloud pairs, dubbed ImLoveNet.
  arXiv  Detail & Related papers  (2022-07-02T13:17:34Z)
• OMNet: Learning Overlapping Mask for Partial-to-Partial Point Cloud Registration [31.108056345511976]
  OMNet is a global feature based iterative network for partial-to-partial point cloud registration. We learn masks in a coarse-to-fine manner to reject non-overlapping regions, which converting the partial-to-partial registration to the registration of the same shapes.
  arXiv  Detail & Related papers  (2021-03-01T11:59:59Z)
• ParaNet: Deep Regular Representation for 3D Point Clouds [62.81379889095186]
  ParaNet is a novel end-to-end deep learning framework for representing 3D point clouds. It converts an irregular 3D point cloud into a regular 2D color image, named point geometry image (PGI) In contrast to conventional regular representation modalities based on multi-view projection and voxelization, the proposed representation is differentiable and reversible.
  arXiv  Detail & Related papers  (2020-12-05T13:19:55Z)
• End-to-End 3D Point Cloud Learning for Registration Task Using Virtual Correspondences [17.70819292121181]
  3D Point cloud registration is still a very challenging topic due to the difficulty in finding the rigid transformation between two point clouds. In this paper, we present an end-to-end deep-learning based approach to resolve the point cloud registration problem.
  arXiv  Detail & Related papers  (2020-11-30T06:55:05Z)
• Graphite: GRAPH-Induced feaTure Extraction for Point Cloud Registration [80.69255347486693]
  We introduce a GRAPH-Induced feaTure Extraction pipeline, a simple yet powerful feature and keypoint detector. We construct a generic graph-based learning scheme to describe point cloud regions and extract salient points. We Reformulate the 3D keypoint pipeline with graph neural networks which allow efficient processing of the point set.
  arXiv  Detail & Related papers  (2020-10-18T19:41:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.