Zero-Shot Point Cloud Registration

• URL: http://arxiv.org/abs/2312.03032v2
• Date: Fri, 8 Dec 2023 17:33:12 GMT
• Title: Zero-Shot Point Cloud Registration
• Authors: Weijie Wang, Guofeng Mei, Bin Ren, Xiaoshui Huang, Fabio Poiesi, Luc Van Gool, Nicu Sebe, Bruno Lepri
• Abstract summary: ZeroReg is the first zero-shot point cloud registration approach that eliminates the need for training on point cloud datasets. The cornerstone of ZeroReg is the novel transfer of image features from keypoints to the point cloud, enriched by aggregating information from 3D geometric neighborhoods. On benchmarks such as 3DMatch, 3DLoMatch, and ScanNet, ZeroReg achieves impressive Recall Ratios (RR) of over 84%, 46%, and 75%, respectively.
• Score: 94.39796531154303
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Learning-based point cloud registration approaches have significantly outperformed their traditional counterparts. However, they typically require extensive training on specific datasets. In this paper, we propose , the first zero-shot point cloud registration approach that eliminates the need for training on point cloud datasets. The cornerstone of ZeroReg is the novel transfer of image features from keypoints to the point cloud, enriched by aggregating information from 3D geometric neighborhoods. Specifically, we extract keypoints and features from 2D image pairs using a frozen pretrained 2D backbone. These features are then projected in 3D, and patches are constructed by searching for neighboring points. We integrate the geometric and visual features of each point using our novel parameter-free geometric decoder. Subsequently, the task of determining correspondences between point clouds is formulated as an optimal transport problem. Extensive evaluations of ZeroReg demonstrate its competitive performance against both traditional and learning-based methods. On benchmarks such as 3DMatch, 3DLoMatch, and ScanNet, ZeroReg achieves impressive Recall Ratios (RR) of over 84%, 46%, and 75%, respectively.

Related papers

• Open-Vocabulary Octree-Graph for 3D Scene Understanding [54.11828083068082]
  Octree-Graph is a novel scene representation for open-vocabulary 3D scene understanding. An adaptive-octree structure is developed that stores semantics and depicts the occupancy of an object adjustably according to its shape.
  arXiv  Detail & Related papers  (2024-11-25T10:14:10Z)
• 3D Focusing-and-Matching Network for Multi-Instance Point Cloud Registration [45.579241614565376]
  We propose a powerful 3D focusing-and-matching network for multi-instance point cloud registration. By using self-attention and cross-attention, we can locate potential matching instances by regressing object centers. Our method achieves a new state-of-the-art performance on the multi-instance point cloud registration task.
  arXiv  Detail & Related papers  (2024-11-12T12:04:44Z)
• SG-PGM: Partial Graph Matching Network with Semantic Geometric Fusion for 3D Scene Graph Alignment and Its Downstream Tasks [14.548198408544032]
  We treat 3D scene graph alignment as a partial graph-matching problem and propose to solve it with a graph neural network. We reuse the geometric features learned by a point cloud registration method and associate the clustered point-level geometric features with the node-level semantic feature. We propose a point-matching rescoring method, that uses the node-wise alignment of the 3D scene graph to reweight the matching candidates from a pre-trained point cloud registration method.
  arXiv  Detail & Related papers  (2024-03-28T15:01:58Z)
• Variational Relational Point Completion Network for Robust 3D Classification [59.80993960827833]
  Vari point cloud completion methods tend to generate global shape skeletons hence lack fine local details. This paper proposes a variational framework, point Completion Network (VRCNet) with two appealing properties. VRCNet shows great generalizability and robustness on real-world point cloud scans.
  arXiv  Detail & Related papers  (2023-04-18T17:03:20Z)
• Robust Point Cloud Registration Framework Based on Deep Graph Matching(TPAMI Version) [13.286247750893681]
  3D point cloud registration is a fundamental problem in computer vision and robotics. We propose a novel deep graph matching-based framework for point cloud registration.
  arXiv  Detail & Related papers  (2022-11-09T06:05:25Z)
• REGTR: End-to-end Point Cloud Correspondences with Transformers [79.52112840465558]
  We conjecture that attention mechanisms can replace the role of explicit feature matching and RANSAC. We propose an end-to-end framework to directly predict the final set of correspondences. Our approach achieves state-of-the-art performance on 3DMatch and ModelNet benchmarks.
  arXiv  Detail & Related papers  (2022-03-28T06:01:00Z)
• SASA: Semantics-Augmented Set Abstraction for Point-based 3D Object Detection [78.90102636266276]
  We propose a novel set abstraction method named Semantics-Augmented Set Abstraction (SASA) Based on the estimated point-wise foreground scores, we then propose a semantics-guided point sampling algorithm to help retain more important foreground points during down-sampling. In practice, SASA shows to be effective in identifying valuable points related to foreground objects and improving feature learning for point-based 3D detection.
  arXiv  Detail & Related papers  (2022-01-06T08:54:47Z)
• Point Cloud Registration using Representative Overlapping Points [10.843159482657303]
  We propose ROPNet, a new deep learning model using Representative Overlapping Points with discriminative features for registration. Specifically, we propose a context-guided module which uses an encoder to extract global features for predicting point overlap score. Experiments over ModelNet40 using noisy and partially overlapping point clouds show that the proposed method outperforms traditional and learning-based methods.
  arXiv  Detail & Related papers  (2021-07-06T12:52:22Z)
• OMNet: Learning Overlapping Mask for Partial-to-Partial Point Cloud Registration [31.108056345511976]
  OMNet is a global feature based iterative network for partial-to-partial point cloud registration. We learn masks in a coarse-to-fine manner to reject non-overlapping regions, which converting the partial-to-partial registration to the registration of the same shapes.
  arXiv  Detail & Related papers  (2021-03-01T11:59:59Z)
• 3D Object Classification on Partial Point Clouds: A Practical Perspective [91.81377258830703]
  A point cloud is a popular shape representation adopted in 3D object classification. This paper introduces a practical setting to classify partial point clouds of object instances under any poses. A novel algorithm in an alignment-classification manner is proposed in this paper.
  arXiv  Detail & Related papers  (2020-12-18T04:00:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.