Semi-Supervised Single-View 3D Reconstruction via Prototype Shape Priors
- URL: http://arxiv.org/abs/2209.15383v1
- Date: Fri, 30 Sep 2022 11:19:25 GMT
- Title: Semi-Supervised Single-View 3D Reconstruction via Prototype Shape Priors
- Authors: Zhen Xing and Hengduo Li and Zuxuan Wu and Yu-Gang Jiang
- Abstract summary: We propose SSP3D, a semi-supervised framework for 3D reconstruction.
We introduce an attention-guided prototype shape prior module for guiding realistic object reconstruction.
Our approach also performs well when transferring to real-world Pix3D datasets under labeling ratios of 10%.
- Score: 79.80916315953374
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The performance of existing single-view 3D reconstruction methods heavily
relies on large-scale 3D annotations. However, such annotations are tedious and
expensive to collect. Semi-supervised learning serves as an alternative way to
mitigate the need for manual labels, but remains unexplored in 3D
reconstruction. Inspired by the recent success of semi-supervised image
classification tasks, we propose SSP3D, a semi-supervised framework for 3D
reconstruction. In particular, we introduce an attention-guided prototype shape
prior module for guiding realistic object reconstruction. We further introduce
a discriminator-guided module to incentivize better shape generation, as well
as a regularizer to tolerate noisy training samples. On the ShapeNet benchmark,
the proposed approach outperforms previous supervised methods by clear margins
under various labeling ratios, (i.e., 1%, 5% , 10% and 20%). Moreover, our
approach also performs well when transferring to real-world Pix3D datasets
under labeling ratios of 10%. We also demonstrate our method could transfer to
novel categories with few novel supervised data. Experiments on the popular
ShapeNet dataset show that our method outperforms the zero-shot baseline by
over 12% and we also perform rigorous ablations and analysis to validate our
approach.
Related papers
- Semi-supervised Single-view 3D Reconstruction via Multi Shape Prior Fusion Strategy and Self-Attention [0.0]
Semi-supervised learning strategies offer an innovative approach to reduce the dependence on labeled data.
We created an innovative framework for 3D reconstruction that distinctively introduces a multi shape prior fusion strategy.
Our framework demonstrated a 3.3% performance improvement over the baseline.
arXiv Detail & Related papers (2024-11-23T02:46:16Z) - Bayesian Self-Training for Semi-Supervised 3D Segmentation [59.544558398992386]
3D segmentation is a core problem in computer vision.
densely labeling 3D point clouds to employ fully-supervised training remains too labor intensive and expensive.
Semi-supervised training provides a more practical alternative, where only a small set of labeled data is given, accompanied by a larger unlabeled set.
arXiv Detail & Related papers (2024-09-12T14:54:31Z) - Robust Geometry-Preserving Depth Estimation Using Differentiable
Rendering [93.94371335579321]
We propose a learning framework that trains models to predict geometry-preserving depth without requiring extra data or annotations.
Comprehensive experiments underscore our framework's superior generalization capabilities.
Our innovative loss functions empower the model to autonomously recover domain-specific scale-and-shift coefficients.
arXiv Detail & Related papers (2023-09-18T12:36:39Z) - A Fusion of Variational Distribution Priors and Saliency Map Replay for Continual 3D Reconstruction [1.2289361708127877]
Single-image 3D reconstruction is a research challenge focused on predicting 3D object shapes from single-view images.
This task requires significant data acquisition to predict both visible and occluded portions of the shape.
We propose a continual learning-based 3D reconstruction method where our goal is to design a model using Variational Priors that can still reconstruct the previously seen classes reasonably even after training on new classes.
arXiv Detail & Related papers (2023-08-17T06:48:55Z) - Weakly Supervised 3D Object Detection with Multi-Stage Generalization [62.96670547848691]
We introduce BA$2$-Det, encompassing pseudo label generation and multi-stage generalization.
We develop three stages of generalization: progressing from complete to partial, static to dynamic, and close to distant.
BA$2$-Det can achieve a 20% relative improvement on the KITTI dataset.
arXiv Detail & Related papers (2023-06-08T17:58:57Z) - On Triangulation as a Form of Self-Supervision for 3D Human Pose
Estimation [57.766049538913926]
Supervised approaches to 3D pose estimation from single images are remarkably effective when labeled data is abundant.
Much of the recent attention has shifted towards semi and (or) weakly supervised learning.
We propose to impose multi-view geometrical constraints by means of a differentiable triangulation and to use it as form of self-supervision during training when no labels are available.
arXiv Detail & Related papers (2022-03-29T19:11:54Z) - Semi-Supervised Adversarial Recognition of Refined Window Structures for
Inverse Procedural Fa\c{c}ade Modeling [17.62526990262815]
This paper proposes a semi-supervised adversarial recognition strategy embedded in inverse procedural modeling.
A simple procedural engine is built inside an existing 3D modeling software, producing fine-grained window geometries.
Experiments using publicly available faccade image datasets reveal that the proposed training strategy can obtain about 10% improvement in classification accuracy.
arXiv Detail & Related papers (2022-01-22T06:34:48Z) - Learning Compositional Shape Priors for Few-Shot 3D Reconstruction [36.40776735291117]
We show that complex encoder-decoder architectures exploit large amounts of per-category data.
We propose three ways to learn a class-specific global shape prior, directly from data.
Experiments on the popular ShapeNet dataset show that our method outperforms a zero-shot baseline by over 40%.
arXiv Detail & Related papers (2021-06-11T14:55:49Z) - Monocular Human Pose and Shape Reconstruction using Part Differentiable
Rendering [53.16864661460889]
Recent works succeed in regression-based methods which estimate parametric models directly through a deep neural network supervised by 3D ground truth.
In this paper, we introduce body segmentation as critical supervision.
To improve the reconstruction with part segmentation, we propose a part-level differentiable part that enables part-based models to be supervised by part segmentation.
arXiv Detail & Related papers (2020-03-24T14:25:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.