Learning Compositional Shape Priors for Few-Shot 3D Reconstruction

• URL: http://arxiv.org/abs/2106.06440v1
• Date: Fri, 11 Jun 2021 14:55:49 GMT
• Title: Learning Compositional Shape Priors for Few-Shot 3D Reconstruction
• Authors: Mateusz Michalkiewicz, Stavros Tsogkas, Sarah Parisot, Mahsa Baktashmotlagh, Anders Eriksson, Eugene Belilovsky
• Abstract summary: We show that complex encoder-decoder architectures exploit large amounts of per-category data. We propose three ways to learn a class-specific global shape prior, directly from data. Experiments on the popular ShapeNet dataset show that our method outperforms a zero-shot baseline by over 40%.
• Score: 36.40776735291117
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The impressive performance of deep convolutional neural networks in single-view 3D reconstruction suggests that these models perform non-trivial reasoning about the 3D structure of the output space. Recent work has challenged this belief, showing that, on standard benchmarks, complex encoder-decoder architectures perform similarly to nearest-neighbor baselines or simple linear decoder models that exploit large amounts of per-category data. However, building large collections of 3D shapes for supervised training is a laborious process; a more realistic and less constraining task is inferring 3D shapes for categories with few available training examples, calling for a model that can successfully generalize to novel object classes. In this work we experimentally demonstrate that naive baselines fail in this few-shot learning setting, in which the network must learn informative shape priors for inference of new categories. We propose three ways to learn a class-specific global shape prior, directly from data. Using these techniques, we are able to capture multi-scale information about the 3D shape, and account for intra-class variability by virtue of an implicit compositional structure. Experiments on the popular ShapeNet dataset show that our method outperforms a zero-shot baseline by over 40%, and the current state-of-the-art by over 10%, in terms of relative performance, in the few-shot setting.12

Related papers

• PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm [114.47216525866435]
  We introduce a novel universal 3D pre-training framework designed to facilitate the acquisition of efficient 3D representation. For the first time, PonderV2 achieves state-of-the-art performance on 11 indoor and outdoor benchmarks, implying its effectiveness.
  arXiv  Detail & Related papers  (2023-10-12T17:59:57Z)
• Common Pets in 3D: Dynamic New-View Synthesis of Real-Life Deformable Categories [80.30216777363057]
  We introduce Common Pets in 3D (CoP3D), a collection of crowd-sourced videos showing around 4,200 distinct pets. At test time, given a small number of video frames of an unseen object, Tracker-NeRF predicts the trajectories of its 3D points and generates new views. Results on CoP3D reveal significantly better non-rigid new-view synthesis performance than existing baselines.
  arXiv  Detail & Related papers  (2022-11-07T22:42:42Z)
• Semi-Supervised Single-View 3D Reconstruction via Prototype Shape Priors [79.80916315953374]
  We propose SSP3D, a semi-supervised framework for 3D reconstruction. We introduce an attention-guided prototype shape prior module for guiding realistic object reconstruction. Our approach also performs well when transferring to real-world Pix3D datasets under labeling ratios of 10%.
  arXiv  Detail & Related papers  (2022-09-30T11:19:25Z)
• Few-shot Single-view 3D Reconstruction with Memory Prior Contrastive Network [18.000566656946475]
  3D reconstruction of novel categories based on few-shot learning is appealing in real-world applications. We present a Memory Prior Contrastive Network (MPCN) that can store shape prior knowledge in a few-shot learning based 3D reconstruction framework.
  arXiv  Detail & Related papers  (2022-07-30T10:49:39Z)
• SurFit: Learning to Fit Surfaces Improves Few Shot Learning on Point Clouds [48.61222927399794]
  SurFit is a simple approach for label efficient learning of 3D shape segmentation networks. It is based on a self-supervised task of decomposing the surface of a 3D shape into geometric primitives.
  arXiv  Detail & Related papers  (2021-12-27T23:55:36Z)
• RandomRooms: Unsupervised Pre-training from Synthetic Shapes and Randomized Layouts for 3D Object Detection [138.2892824662943]
  A promising solution is to make better use of the synthetic dataset, which consists of CAD object models, to boost the learning on real datasets. Recent work on 3D pre-training exhibits failure when transfer features learned on synthetic objects to other real-world applications. In this work, we put forward a new method called RandomRooms to accomplish this objective.
  arXiv  Detail & Related papers  (2021-08-17T17:56:12Z)
• Unsupervised Learning of 3D Object Categories from Videos in the Wild [75.09720013151247]
  We focus on learning a model from multiple views of a large collection of object instances. We propose a new neural network design, called warp-conditioned ray embedding (WCR), which significantly improves reconstruction. Our evaluation demonstrates performance improvements over several deep monocular reconstruction baselines on existing benchmarks.
  arXiv  Detail & Related papers  (2021-03-30T17:57:01Z)
• LightSAL: Lightweight Sign Agnostic Learning for Implicit Surface Representation [5.1135133995376085]
  This work proposes LightSAL, a novel deep convolutional architecture for learning 3D shapes. Experiments are based on the D-Faust dataset that contains 41k 3D scans of human shapes.
  arXiv  Detail & Related papers  (2021-03-26T05:50:14Z)
• Few-Shot Single-View 3-D Object Reconstruction with Compositional Priors [30.262308825799167]
  We show that complex encoder-decoder architectures perform similarly to nearest-neighbor baselines in standard benchmarks. We propose three approaches that efficiently integrate a class prior into a 3D reconstruction model.
  arXiv  Detail & Related papers  (2020-04-14T04:53:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.