Benefit from Reference: Retrieval-Augmented Cross-modal Point Cloud Completion

• URL: http://arxiv.org/abs/2507.14485v1
• Date: Sat, 19 Jul 2025 04:57:41 GMT
• Title: Benefit from Reference: Retrieval-Augmented Cross-modal Point Cloud Completion
• Authors: Hongye Hou, Liu Zhan, Yang Yang,
• Abstract summary: We propose a novel retrieval-augmented point cloud completion framework.<n>The core idea is to incorporate cross-modal retrieval into completion task to learn structural prior information.<n>Our method shows its effectiveness in generating fine-grained point clouds.
• Score: 3.2899630403451985
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Completing the whole 3D structure based on an incomplete point cloud is a challenging task, particularly when the residual point cloud lacks typical structural characteristics. Recent methods based on cross-modal learning attempt to introduce instance images to aid the structure feature learning. However, they still focus on each particular input class, limiting their generation abilities. In this work, we propose a novel retrieval-augmented point cloud completion framework. The core idea is to incorporate cross-modal retrieval into completion task to learn structural prior information from similar reference samples. Specifically, we design a Structural Shared Feature Encoder (SSFE) to jointly extract cross-modal features and reconstruct reference features as priors. Benefiting from a dual-channel control gate in the encoder, relevant structural features in the reference sample are enhanced and irrelevant information interference is suppressed. In addition, we propose a Progressive Retrieval-Augmented Generator (PRAG) that employs a hierarchical feature fusion mechanism to integrate reference prior information with input features from global to local. Through extensive evaluations on multiple datasets and real-world scenes, our method shows its effectiveness in generating fine-grained point clouds, as well as its generalization capability in handling sparse data and unseen categories.

Related papers

• RefComp: A Reference-guided Unified Framework for Unpaired Point Cloud Completion [53.28542050638217]
  The unpaired point cloud completion task aims to complete a partial point cloud by using models trained with no ground truth.<n>Existing unpaired point cloud completion methods are class-aware, i.e., a separate model is needed for each object class.<n>We propose a novel unpaired point cloud completion framework, namely the Reference-guided Completion (RefComp) framework.
  arXiv  Detail & Related papers  (2025-04-18T16:40:16Z)
• Human as Points: Explicit Point-based 3D Human Reconstruction from Single-view RGB Images [71.91424164693422]
  We introduce an explicit point-based human reconstruction framework called HaP.<n>Our approach is featured by fully-explicit point cloud estimation, manipulation, generation, and refinement in the 3D geometric space.<n>Our results may indicate a paradigm rollback to the fully-explicit and geometry-centric algorithm design.
  arXiv  Detail & Related papers  (2023-11-06T05:52:29Z)
• Point Cloud Completion Guided by Prior Knowledge via Causal Inference [19.935868881427226]
  We propose a novel approach to point cloud completion task called Point-PC. Point-PC uses a memory network to retrieve shape priors and designs a causal inference model to filter missing shape information. Experimental results on the ShapeNet-55, PCN, and KITTI datasets demonstrate that Point-PC outperforms the state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-28T16:33:35Z)
• FBNet: Feedback Network for Point Cloud Completion [35.89264923599902]
  We propose a novel Feedback Network (FBNet) for point cloud completion, in which present features are efficiently refined by rerouting subsequent fine-grained ones. The main challenge of building feedback connections is the mismatching between present and subsequent features. To address this, the elaborately designed point Cross Transformer exploits efficient information from feedback features via cross attention strategy.
  arXiv  Detail & Related papers  (2022-10-08T09:12:37Z)
• MAPLE: Masked Pseudo-Labeling autoEncoder for Semi-supervised Point Cloud Action Recognition [160.49403075559158]
  We propose a Masked Pseudo-Labeling autoEncoder (textbfMAPLE) framework for point cloud action recognition. In particular, we design a novel and efficient textbfDecoupled textbfspatial-textbftemporal TranstextbfFormer (textbfDestFormer) as the backbone of MAPLE. MAPLE achieves superior results on three public benchmarks and outperforms the state-of-the-art method by 8.08% accuracy on the MSR-Action3
  arXiv  Detail & Related papers  (2022-09-01T12:32:40Z)
• Point cloud completion on structured feature map with feedback network [28.710494879042002]
  We propose FSNet, a feature structuring module that can adaptively aggregate point-wise features into a 2D structured feature map. A 2D convolutional neural network is adopted to decode feature maps from FSNet into a coarse and complete point cloud. A point cloud upsampling network is used to generate dense point cloud from the partial input and the coarse intermediate output.
  arXiv  Detail & Related papers  (2022-02-17T10:59:40Z)
• SRPCN: Structure Retrieval based Point Completion Network [9.456072124396231]
  We propose a Structure Retrieval based Point Completion Network (SRPCN) It first uses k-means clustering to extract structure points and disperse them into distributions, and then KL Divergence is used as a metric to find the complete structure point cloud. Experiments show that our method can generate more authentic results and has a stronger generalization ability.
  arXiv  Detail & Related papers  (2022-02-06T01:20:50Z)
• PC-RGNN: Point Cloud Completion and Graph Neural Network for 3D Object Detection [57.49788100647103]
  LiDAR-based 3D object detection is an important task for autonomous driving. Current approaches suffer from sparse and partial point clouds of distant and occluded objects. In this paper, we propose a novel two-stage approach, namely PC-RGNN, dealing with such challenges by two specific solutions.
  arXiv  Detail & Related papers  (2020-12-18T18:06:43Z)
• Cascaded Refinement Network for Point Cloud Completion with Self-supervision [74.80746431691938]
  We introduce a two-branch network for shape completion. The first branch is a cascaded shape completion sub-network to synthesize complete objects. The second branch is an auto-encoder to reconstruct the original partial input.
  arXiv  Detail & Related papers  (2020-10-17T04:56:22Z)
• Detail Preserved Point Cloud Completion via Separated Feature Aggregation [26.566021924980706]
  Point cloud shape completion is a challenging problem in 3D vision and robotics. We propose two different feature aggregation strategies, named global & local feature aggregation(GLFA) and residual feature aggregation(RFA) Our proposed network outperforms current state-of-the art methods especially on detail preservation.
  arXiv  Detail & Related papers  (2020-07-05T16:11:55Z)
• Point Cloud Completion by Skip-attention Network with Hierarchical Folding [61.59710288271434]
  We propose Skip-Attention Network (SA-Net) for 3D point cloud completion. First, we propose a skip-attention mechanism to effectively exploit the local structure details of incomplete point clouds. Second, in order to fully utilize the selected geometric information encoded by skip-attention mechanism at different resolutions, we propose a novel structure-preserving decoder.
  arXiv  Detail & Related papers  (2020-05-08T06:23:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.