MFM-Net: Unpaired Shape Completion Network with Multi-stage Feature Matching

• URL: http://arxiv.org/abs/2111.11976v1
• Date: Tue, 23 Nov 2021 16:10:06 GMT
• Title: MFM-Net: Unpaired Shape Completion Network with Multi-stage Feature Matching
• Authors: Zhen Cao, Wenxiao Zhang, Xin Wen, Zhen Dong, Yu-shen Liu, Bisheng Yang
• Abstract summary: We propose a novel unpaired shape completion network, named MFM-Net, which decomposes the learning of geometric correspondence into multi-stages. MFM-Net makes use of a more comprehensive understanding to establish the geometric correspondence between complete and incomplete shapes.
• Score: 38.63975659511946
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unpaired 3D object completion aims to predict a complete 3D shape from an incomplete input without knowing the correspondence between the complete and incomplete shapes during training. To build the correspondence between two data modalities, previous methods usually apply adversarial training to match the global shape features extracted by the encoder. However, this ignores the correspondence between multi-scaled geometric information embedded in the pyramidal hierarchy of the decoder, which makes previous methods struggle to generate high-quality complete shapes. To address this problem, we propose a novel unpaired shape completion network, named MFM-Net, using multi-stage feature matching, which decomposes the learning of geometric correspondence into multi-stages throughout the hierarchical generation process in the point cloud decoder. Specifically, MFM-Net adopts a dual path architecture to establish multiple feature matching channels in different layers of the decoder, which is then combined with the adversarial learning to merge the distribution of features from complete and incomplete modalities. In addition, a refinement is applied to enhance the details. As a result, MFM-Net makes use of a more comprehensive understanding to establish the geometric correspondence between complete and incomplete shapes in a local-to-global perspective, which enables more detailed geometric inference for generating high-quality complete shapes. We conduct comprehensive experiments on several datasets, and the results show that our method outperforms previous methods of unpaired point cloud completion with a large margin.

Related papers

• 360 Layout Estimation via Orthogonal Planes Disentanglement and Multi-view Geometric Consistency Perception [56.84921040837699]
  Existing panoramic layout estimation solutions tend to recover room boundaries from a vertically compressed sequence, yielding imprecise results. We propose an orthogonal plane disentanglement network (termed DOPNet) to distinguish ambiguous semantics. We also present an unsupervised adaptation technique tailored for horizon-depth and ratio representations. Our solution outperforms other SoTA models on both monocular layout estimation and multi-view layout estimation tasks.
  arXiv  Detail & Related papers  (2023-12-26T12:16:03Z)
• DiffComplete: Diffusion-based Generative 3D Shape Completion [114.43353365917015]
  We introduce a new diffusion-based approach for shape completion on 3D range scans. We strike a balance between realism, multi-modality, and high fidelity. DiffComplete sets a new SOTA performance on two large-scale 3D shape completion benchmarks.
  arXiv  Detail & Related papers  (2023-06-28T16:07:36Z)
• Self-Supervised Learning for Multimodal Non-Rigid 3D Shape Matching [15.050801537501462]
  We introduce a self-supervised multimodal learning strategy that combines mesh-based functional map regularisation with a contrastive loss that couples mesh and point cloud data. Our shape matching approach allows to obtain intramodal correspondences for triangle meshes, complete point clouds, and partially observed point clouds. We demonstrate that our method achieves state-of-the-art results on several challenging benchmark datasets.
  arXiv  Detail & Related papers  (2023-03-20T09:47:02Z)
• PointMCD: Boosting Deep Point Cloud Encoders via Multi-view Cross-modal Distillation for 3D Shape Recognition [55.38462937452363]
  We propose a unified multi-view cross-modal distillation architecture, including a pretrained deep image encoder as the teacher and a deep point encoder as the student. By pair-wise aligning multi-view visual and geometric descriptors, we can obtain more powerful deep point encoders without exhausting and complicated network modification.
  arXiv  Detail & Related papers  (2022-07-07T07:23:20Z)
• DFC: Deep Feature Consistency for Robust Point Cloud Registration [0.4724825031148411]
  We present a novel learning-based alignment network for complex alignment scenes. We validate our approach on the 3DMatch dataset and the KITTI odometry dataset.
  arXiv  Detail & Related papers  (2021-11-15T08:27:21Z)
• PMP-Net: Point Cloud Completion by Learning Multi-step Point Moving Paths [54.459879603473034]
  We design a novel neural network, named PMP-Net, to mimic the behavior of an earth mover. It moves each point of the incomplete input to complete the point cloud, where the total distance of point moving paths should be shortest. It learns a strict and unique correspondence on point-level, which can capture the detailed topology and structure relationships between the incomplete shape and the complete target.
  arXiv  Detail & Related papers  (2020-12-07T01:34:38Z)
• DmifNet:3D Shape Reconstruction Based on Dynamic Multi-Branch Information Fusion [14.585272577456472]
  3D object reconstruction from a single-view image is a long-standing challenging problem. Previous work was difficult to accurately reconstruct 3D shapes with a complex topology which has rich details at the edges and corners. We propose a Dynamic Multi-branch Information Fusion Network (DmifNet) which can recover a high-fidelity 3D shape of arbitrary topology from a 2D image.
  arXiv  Detail & Related papers  (2020-11-21T11:31:27Z)
• Detail Preserved Point Cloud Completion via Separated Feature Aggregation [26.566021924980706]
  Point cloud shape completion is a challenging problem in 3D vision and robotics. We propose two different feature aggregation strategies, named global & local feature aggregation(GLFA) and residual feature aggregation(RFA) Our proposed network outperforms current state-of-the art methods especially on detail preservation.
  arXiv  Detail & Related papers  (2020-07-05T16:11:55Z)
• Cascaded Refinement Network for Point Cloud Completion [74.80746431691938]
  We propose a cascaded refinement network together with a coarse-to-fine strategy to synthesize the detailed object shapes. Considering the local details of partial input with the global shape information together, we can preserve the existing details in the incomplete point set. We also design a patch discriminator that guarantees every local area has the same pattern with the ground truth to learn the complicated point distribution.
  arXiv  Detail & Related papers  (2020-04-07T13:03:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.