Photorealistic Monocular 3D Reconstruction of Humans Wearing Clothing
- URL: http://arxiv.org/abs/2204.08906v1
- Date: Tue, 19 Apr 2022 14:06:16 GMT
- Title: Photorealistic Monocular 3D Reconstruction of Humans Wearing Clothing
- Authors: Thiemo Alldieck, Mihai Zanfir, Cristian Sminchisescu
- Abstract summary: We present PHORHUM, a novel, end-to-end trainable, deep neural network methodology for photorealistic 3D human reconstruction given just a monocular RGB image.
Our pixel-aligned method estimates detailed 3D geometry and, for the first time, the unshaded surface color together with the scene illumination.
- Score: 41.34640834483265
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We present PHORHUM, a novel, end-to-end trainable, deep neural network
methodology for photorealistic 3D human reconstruction given just a monocular
RGB image. Our pixel-aligned method estimates detailed 3D geometry and, for the
first time, the unshaded surface color together with the scene illumination.
Observing that 3D supervision alone is not sufficient for high fidelity color
reconstruction, we introduce patch-based rendering losses that enable reliable
color reconstruction on visible parts of the human, and detailed and plausible
color estimation for the non-visible parts. Moreover, our method specifically
addresses methodological and practical limitations of prior work in terms of
representing geometry, albedo, and illumination effects, in an end-to-end model
where factors can be effectively disentangled. In extensive experiments, we
demonstrate the versatility and robustness of our approach. Our
state-of-the-art results validate the method qualitatively and for different
metrics, for both geometric and color reconstruction.
Related papers
- ANIM: Accurate Neural Implicit Model for Human Reconstruction from a single RGB-D image [40.03212588672639]
ANIM is a novel method that reconstructs arbitrary 3D human shapes from single-view RGB-D images with an unprecedented level of accuracy.
Our model learns geometric details from both pixel-aligned and voxel-aligned features to leverage depth information.
Experiments demonstrate that ANIM outperforms state-of-the-art works that use RGB, surface normals, point cloud or RGB-D data as input.
arXiv Detail & Related papers (2024-03-15T14:45:38Z) - Multi-View Neural Surface Reconstruction with Structured Light [7.709526244898887]
Three-dimensional (3D) object reconstruction based on differentiable rendering (DR) is an active research topic in computer vision.
We introduce active sensing with structured light (SL) into multi-view 3D object reconstruction based on DR to learn the unknown geometry and appearance of arbitrary scenes and camera poses.
Our method realizes high reconstruction accuracy in the textureless region and reduces efforts for camera pose calibration.
arXiv Detail & Related papers (2022-11-22T03:10:46Z) - ReFu: Refine and Fuse the Unobserved View for Detail-Preserving
Single-Image 3D Human Reconstruction [31.782985891629448]
Single-image 3D human reconstruction aims to reconstruct the 3D textured surface of the human body given a single image.
We propose ReFu, a coarse-to-fine approach that refines the projected backside view image and fuses the refined image to predict the final human body.
arXiv Detail & Related papers (2022-11-09T09:14:11Z) - SIRA: Relightable Avatars from a Single Image [19.69326772087838]
We introduce SIRA, a method which reconstructs human head avatars with high fidelity geometry and factorized lights and surface materials.
Our key ingredients are two data-driven statistical models based on neural fields that resolve the ambiguities of single-view 3D surface reconstruction and appearance factorization.
arXiv Detail & Related papers (2022-09-07T09:47:46Z) - Neural 3D Reconstruction in the Wild [86.6264706256377]
We introduce a new method that enables efficient and accurate surface reconstruction from Internet photo collections.
We present a new benchmark and protocol for evaluating reconstruction performance on such in-the-wild scenes.
arXiv Detail & Related papers (2022-05-25T17:59:53Z) - AvatarMe++: Facial Shape and BRDF Inference with Photorealistic
Rendering-Aware GANs [119.23922747230193]
We introduce the first method that is able to reconstruct render-ready 3D facial geometry and BRDF from a single "in-the-wild" image.
Our method outperforms the existing arts by a significant margin and reconstructs high-resolution 3D faces from a single low-resolution image.
arXiv Detail & Related papers (2021-12-11T11:36:30Z) - 3D Human Texture Estimation from a Single Image with Transformers [106.6320286821364]
We propose a Transformer-based framework for 3D human texture estimation from a single image.
We also propose a mask-fusion strategy to combine the advantages of the RGB-based and texture-flow-based models.
arXiv Detail & Related papers (2021-09-06T16:00:20Z) - SIDER: Single-Image Neural Optimization for Facial Geometric Detail
Recovery [54.64663713249079]
SIDER is a novel photometric optimization method that recovers detailed facial geometry from a single image in an unsupervised manner.
In contrast to prior work, SIDER does not rely on any dataset priors and does not require additional supervision from multiple views, lighting changes or ground truth 3D shape.
arXiv Detail & Related papers (2021-08-11T22:34:53Z) - From Points to Multi-Object 3D Reconstruction [71.17445805257196]
We propose a method to detect and reconstruct multiple 3D objects from a single RGB image.
A keypoint detector localizes objects as center points and directly predicts all object properties, including 9-DoF bounding boxes and 3D shapes.
The presented approach performs lightweight reconstruction in a single-stage, it is real-time capable, fully differentiable and end-to-end trainable.
arXiv Detail & Related papers (2020-12-21T18:52:21Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.