Few-shot Single-view 3D Reconstruction with Memory Prior Contrastive Network

• URL: http://arxiv.org/abs/2208.00183v1
• Date: Sat, 30 Jul 2022 10:49:39 GMT
• Title: Few-shot Single-view 3D Reconstruction with Memory Prior Contrastive Network
• Authors: Zhen Xing and Yijiang Chen and Zhixin Ling and Xiangdong Zhou and Yu Xiang
• Abstract summary: 3D reconstruction of novel categories based on few-shot learning is appealing in real-world applications. We present a Memory Prior Contrastive Network (MPCN) that can store shape prior knowledge in a few-shot learning based 3D reconstruction framework.
• Score: 18.000566656946475
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D reconstruction of novel categories based on few-shot learning is appealing in real-world applications and attracts increasing research interests. Previous approaches mainly focus on how to design shape prior models for different categories. Their performance on unseen categories is not very competitive. In this paper, we present a Memory Prior Contrastive Network (MPCN) that can store shape prior knowledge in a few-shot learning based 3D reconstruction framework. With the shape memory, a multi-head attention module is proposed to capture different parts of a candidate shape prior and fuse these parts together to guide 3D reconstruction of novel categories. Besides, we introduce a 3D-aware contrastive learning method, which can not only complement the retrieval accuracy of memory network, but also better organize image features for downstream tasks. Compared with previous few-shot 3D reconstruction methods, MPCN can handle the inter-class variability without category annotations. Experimental results on a benchmark synthetic dataset and the Pascal3D+ real-world dataset show that our model outperforms the current state-of-the-art methods significantly.

Related papers

• Split-and-Fit: Learning B-Reps via Structure-Aware Voronoi Partitioning [50.684254969269546]
  We introduce a novel method for acquiring boundary representations (B-Reps) of 3D CAD models. We apply a spatial partitioning to derive a single primitive within each partition. We show that our network, coined NVD-Net for neural Voronoi diagrams, can effectively learn Voronoi partitions for CAD models from training data.
  arXiv  Detail & Related papers  (2024-06-07T21:07:49Z)
• Part123: Part-aware 3D Reconstruction from a Single-view Image [54.589723979757515]
  Part123 is a novel framework for part-aware 3D reconstruction from a single-view image. We introduce contrastive learning into a neural rendering framework to learn a part-aware feature space. A clustering-based algorithm is also developed to automatically derive 3D part segmentation results from the reconstructed models.
  arXiv  Detail & Related papers  (2024-05-27T07:10:21Z)
• A Fusion of Variational Distribution Priors and Saliency Map Replay for Continual 3D Reconstruction [1.3812010983144802]
  Single-image 3D reconstruction is a research challenge focused on predicting 3D object shapes from single-view images. This task requires significant data acquisition to predict both visible and occluded portions of the shape. We propose a continual learning-based 3D reconstruction method where our goal is to design a model using Variational Priors that can still reconstruct the previously seen classes reasonably even after training on new classes.
  arXiv  Detail & Related papers  (2023-08-17T06:48:55Z)
• Semi-Supervised Single-View 3D Reconstruction via Prototype Shape Priors [79.80916315953374]
  We propose SSP3D, a semi-supervised framework for 3D reconstruction. We introduce an attention-guided prototype shape prior module for guiding realistic object reconstruction. Our approach also performs well when transferring to real-world Pix3D datasets under labeling ratios of 10%.
  arXiv  Detail & Related papers  (2022-09-30T11:19:25Z)
• Single-view 3D Mesh Reconstruction for Seen and Unseen Categories [69.29406107513621]
  Single-view 3D Mesh Reconstruction is a fundamental computer vision task that aims at recovering 3D shapes from single-view RGB images. This paper tackles Single-view 3D Mesh Reconstruction, to study the model generalization on unseen categories. We propose an end-to-end two-stage network, GenMesh, to break the category boundaries in reconstruction.
  arXiv  Detail & Related papers  (2022-08-04T14:13:35Z)
• Pre-train, Self-train, Distill: A simple recipe for Supersizing 3D Reconstruction [47.38670633513938]
  We learn a unified model for single-view 3D reconstruction of objects from hundreds of semantic categories. Our work relies on segmented image collections for learning 3D of generic categories.
  arXiv  Detail & Related papers  (2022-04-07T17:59:25Z)
• Learning Compositional Shape Priors for Few-Shot 3D Reconstruction [36.40776735291117]
  We show that complex encoder-decoder architectures exploit large amounts of per-category data. We propose three ways to learn a class-specific global shape prior, directly from data. Experiments on the popular ShapeNet dataset show that our method outperforms a zero-shot baseline by over 40%.
  arXiv  Detail & Related papers  (2021-06-11T14:55:49Z)
• Unsupervised Learning of 3D Object Categories from Videos in the Wild [75.09720013151247]
  We focus on learning a model from multiple views of a large collection of object instances. We propose a new neural network design, called warp-conditioned ray embedding (WCR), which significantly improves reconstruction. Our evaluation demonstrates performance improvements over several deep monocular reconstruction baselines on existing benchmarks.
  arXiv  Detail & Related papers  (2021-03-30T17:57:01Z)
• Few-Shot Single-View 3-D Object Reconstruction with Compositional Priors [30.262308825799167]
  We show that complex encoder-decoder architectures perform similarly to nearest-neighbor baselines in standard benchmarks. We propose three approaches that efficiently integrate a class prior into a 3D reconstruction model.
  arXiv  Detail & Related papers  (2020-04-14T04:53:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.