Self-supervised 3D Point Cloud Completion via Multi-view Adversarial Learning

• URL: http://arxiv.org/abs/2407.09786v2
• Date: Wed, 6 Nov 2024 14:22:28 GMT
• Title: Self-supervised 3D Point Cloud Completion via Multi-view Adversarial Learning
• Authors: Lintai Wu, Xianjing Cheng, Yong Xu, Huanqiang Zeng, Junhui Hou,
• Abstract summary: We propose MAL-SPC, a framework that effectively leverages both object-level and category-specific geometric similarities to complete missing structures. Our MAL-SPC does not require any 3D complete supervision and only necessitates a single partial point cloud for each object.
• Score: 61.14132533712537
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In real-world scenarios, scanned point clouds are often incomplete due to occlusion issues. The task of self-supervised point cloud completion involves reconstructing missing regions of these incomplete objects without the supervision of complete ground truth. Current self-supervised methods either rely on multiple views of partial observations for supervision or overlook the intrinsic geometric similarity that can be identified and utilized from the given partial point clouds. In this paper, we propose MAL-SPC, a framework that effectively leverages both object-level and category-specific geometric similarities to complete missing structures. Our MAL-SPC does not require any 3D complete supervision and only necessitates a single partial point cloud for each object. Specifically, we first introduce a Pattern Retrieval Network to retrieve similar position and curvature patterns between the partial input and the predicted shape, then leverage these similarities to densify and refine the reconstructed results. Additionally, we render the reconstructed complete shape into multi-view depth maps and design an adversarial learning module to learn the geometry of the target shape from category-specific single-view depth images. To achieve anisotropic rendering, we design a density-aware radius estimation algorithm to improve the quality of the rendered images. Our MAL-SPC yields the best results compared to current state-of-the-art methods.We will make the source code publicly available at \url{https://github.com/ltwu6/malspc

Related papers

• SVDFormer: Complementing Point Cloud via Self-view Augmentation and Self-structure Dual-generator [30.483163963846206]
  We propose a novel network, SVDFormer, to tackle two specific challenges in point cloud completion. We first design a Self-view Fusion Network that leverages multiple-view depth image information to observe incomplete self-shape. We then introduce a refinement module, called Self-structure Dual-generator, in which we incorporate learned shape priors and geometric self-similarities for producing new points.
  arXiv  Detail & Related papers  (2023-07-17T13:55:31Z)
• Variational Relational Point Completion Network for Robust 3D Classification [59.80993960827833]
  Vari point cloud completion methods tend to generate global shape skeletons hence lack fine local details. This paper proposes a variational framework, point Completion Network (VRCNet) with two appealing properties. VRCNet shows great generalizability and robustness on real-world point cloud scans.
  arXiv  Detail & Related papers  (2023-04-18T17:03:20Z)
• Leveraging Single-View Images for Unsupervised 3D Point Cloud Completion [53.93172686610741]
  Cross-PCC is an unsupervised point cloud completion method without requiring any 3D complete point clouds. To take advantage of the complementary information from 2D images, we use a single-view RGB image to extract 2D features. Our method even achieves comparable performance to some supervised methods.
  arXiv  Detail & Related papers  (2022-12-01T15:11:21Z)
• Single-view 3D Mesh Reconstruction for Seen and Unseen Categories [69.29406107513621]
  Single-view 3D Mesh Reconstruction is a fundamental computer vision task that aims at recovering 3D shapes from single-view RGB images. This paper tackles Single-view 3D Mesh Reconstruction, to study the model generalization on unseen categories. We propose an end-to-end two-stage network, GenMesh, to break the category boundaries in reconstruction.
  arXiv  Detail & Related papers  (2022-08-04T14:13:35Z)
• Self-supervised Point Cloud Completion on Real Traffic Scenes via Scene-concerned Bottom-up Mechanism [14.255659581428333]
  Point cloud completion aims to refer the complete shapes for incomplete 3D scans of objects. Current deep learning-based approaches rely on large-scale complete shapes in the training process. We propose a self-supervised point cloud completion method (TraPCC) for vehicles in real traffic scenes without any complete data.
  arXiv  Detail & Related papers  (2022-03-20T14:42:37Z)
• Variational Relational Point Completion Network [41.98957577398084]
  Existing point cloud completion methods generate global shape skeletons and lack fine local details. This paper proposes Variational point Completion network (VRCNet) with two appealing properties. VRCNet shows greatizability and robustness on real-world point cloud scans.
  arXiv  Detail & Related papers  (2021-04-20T17:53:40Z)
• 3D Object Classification on Partial Point Clouds: A Practical Perspective [91.81377258830703]
  A point cloud is a popular shape representation adopted in 3D object classification. This paper introduces a practical setting to classify partial point clouds of object instances under any poses. A novel algorithm in an alignment-classification manner is proposed in this paper.
  arXiv  Detail & Related papers  (2020-12-18T04:00:56Z)
• RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction [19.535169371240073]
  We introduce RfD-Net that jointly detects and reconstructs dense object surfaces directly from point clouds. We decouple the instance reconstruction into global object localization and local shape prediction. Our approach consistently outperforms the state-of-the-arts and improves over 11 of mesh IoU in object reconstruction.
  arXiv  Detail & Related papers  (2020-11-30T12:58:05Z)
• Point Cloud Completion by Skip-attention Network with Hierarchical Folding [61.59710288271434]
  We propose Skip-Attention Network (SA-Net) for 3D point cloud completion. First, we propose a skip-attention mechanism to effectively exploit the local structure details of incomplete point clouds. Second, in order to fully utilize the selected geometric information encoded by skip-attention mechanism at different resolutions, we propose a novel structure-preserving decoder.
  arXiv  Detail & Related papers  (2020-05-08T06:23:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.