DuInNet: Dual-Modality Feature Interaction for Point Cloud Completion

• URL: http://arxiv.org/abs/2407.07374v1
• Date: Wed, 10 Jul 2024 05:19:40 GMT
• Title: DuInNet: Dual-Modality Feature Interaction for Point Cloud Completion
• Authors: Xinpu Liu, Baolin Hou, Hanyun Wang, Ke Xu, Jianwei Wan, Yulan Guo,
• Abstract summary: We contribute a large-scale multimodal point cloud completion benchmark ModelNet-MPC with richer shape categories and more diverse test data. Besides the fully supervised point cloud completion task, two additional tasks including denoising completion and zero-shot learning completion are proposed. Experiments on the ShapeNet-ViPC and ModelNet-MPC benchmarks demonstrate that DuInNet exhibits superiority, robustness and transfer ability in all completion tasks over state-of-the-art methods.
• Score: 33.023714580710504
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To further promote the development of multimodal point cloud completion, we contribute a large-scale multimodal point cloud completion benchmark ModelNet-MPC with richer shape categories and more diverse test data, which contains nearly 400,000 pairs of high-quality point clouds and rendered images of 40 categories. Besides the fully supervised point cloud completion task, two additional tasks including denoising completion and zero-shot learning completion are proposed in ModelNet-MPC, to simulate real-world scenarios and verify the robustness to noise and the transfer ability across categories of current methods. Meanwhile, considering that existing multimodal completion pipelines usually adopt a unidirectional fusion mechanism and ignore the shape prior contained in the image modality, we propose a Dual-Modality Feature Interaction Network (DuInNet) in this paper. DuInNet iteratively interacts features between point clouds and images to learn both geometric and texture characteristics of shapes with the dual feature interactor. To adapt to specific tasks such as fully supervised, denoising, and zero-shot learning point cloud completions, an adaptive point generator is proposed to generate complete point clouds in blocks with different weights for these two modalities. Extensive experiments on the ShapeNet-ViPC and ModelNet-MPC benchmarks demonstrate that DuInNet exhibits superiority, robustness and transfer ability in all completion tasks over state-of-the-art methods. The code and dataset will be available soon.

Related papers

• DMF-Net: Image-Guided Point Cloud Completion with Dual-Channel Modality Fusion and Shape-Aware Upsampling Transformer [7.210417508954435]
  We propose a novel dual-channel modality fusion network for image-guided point cloud completion. In the first stage, DMF-Net takes a partial point cloud and corresponding image as input to recover a coarse point cloud. In the second stage, the coarse point cloud will be upsampled twice with shape-aware upsampling transformer to get the dense and complete point cloud.
  arXiv  Detail & Related papers  (2024-06-25T07:08:19Z)
• P2M2-Net: Part-Aware Prompt-Guided Multimodal Point Cloud Completion [6.407066306127476]
  Inferring missing regions from severely occluded point clouds is highly challenging. We propose a novel prompt-guided point cloud completion framework, coined P2M2-Net. Given an input partial point cloud and a text prompt describing the part-aware information, our Transformer-based completion network can efficiently fuse the multimodal features.
  arXiv  Detail & Related papers  (2023-12-29T14:11:45Z)
• Point Cloud Pre-training with Diffusion Models [62.12279263217138]
  We propose a novel pre-training method called Point cloud Diffusion pre-training (PointDif) PointDif achieves substantial improvement across various real-world datasets for diverse downstream tasks such as classification, segmentation and detection.
  arXiv  Detail & Related papers  (2023-11-25T08:10:05Z)
• Rotation-Invariant Completion Network [8.023732679237021]
  Real-world point clouds usually suffer from incompleteness and display different poses. Current point cloud completion methods excel in reproducing complete point clouds with consistent poses as seen in the training set. We propose a network named Rotation-Invariant Completion Network (RICNet), which consists of two parts: a Dual Pipeline Completion Network (DPCNet) and an enhancing module.
  arXiv  Detail & Related papers  (2023-08-23T07:58:20Z)
• Variational Relational Point Completion Network for Robust 3D Classification [59.80993960827833]
  Vari point cloud completion methods tend to generate global shape skeletons hence lack fine local details. This paper proposes a variational framework, point Completion Network (VRCNet) with two appealing properties. VRCNet shows great generalizability and robustness on real-world point cloud scans.
  arXiv  Detail & Related papers  (2023-04-18T17:03:20Z)
• Point2Vec for Self-Supervised Representation Learning on Point Clouds [66.53955515020053]
  We extend data2vec to the point cloud domain and report encouraging results on several downstream tasks. We propose point2vec, which unleashes the full potential of data2vec-like pre-training on point clouds.
  arXiv  Detail & Related papers  (2023-03-29T10:08:29Z)
• AdaPoinTr: Diverse Point Cloud Completion with Adaptive Geometry-Aware Transformers [94.11915008006483]
  We present a new method that reformulates point cloud completion as a set-to-set translation problem. We design a new model, called PoinTr, which adopts a Transformer encoder-decoder architecture for point cloud completion. Our method attains 6.53 CD on PCN, 0.81 CD on ShapeNet-55 and 0.392 MMD on real-world KITTI.
  arXiv  Detail & Related papers  (2023-01-11T16:14:12Z)
• A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completion [69.32451612060214]
  Real-scanned 3D point clouds are often incomplete, and it is important to recover complete point clouds for downstream applications. Most existing point cloud completion methods use Chamfer Distance (CD) loss for training. We propose a novel Point Diffusion-Refinement (PDR) paradigm for point cloud completion.
  arXiv  Detail & Related papers  (2021-12-07T06:59:06Z)
• Voint Cloud: Multi-View Point Cloud Representation for 3D Understanding [80.04281842702294]
  We introduce the concept of the multi-view point cloud (Voint cloud) representing each 3D point as a set of features extracted from several view-points. This novel 3D Voint cloud representation combines the compactness of 3D point cloud representation with the natural view-awareness of multi-view representation. We deploy a Voint neural network (VointNet) with a theoretically established functional form to learn representations in the Voint space.
  arXiv  Detail & Related papers  (2021-11-30T13:08:19Z)
• Variational Relational Point Completion Network [41.98957577398084]
  Existing point cloud completion methods generate global shape skeletons and lack fine local details. This paper proposes Variational point Completion network (VRCNet) with two appealing properties. VRCNet shows greatizability and robustness on real-world point cloud scans.
  arXiv  Detail & Related papers  (2021-04-20T17:53:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.