Neural Haircut: Prior-Guided Strand-Based Hair Reconstruction
- URL: http://arxiv.org/abs/2306.05872v2
- Date: Mon, 12 Jun 2023 10:31:38 GMT
- Title: Neural Haircut: Prior-Guided Strand-Based Hair Reconstruction
- Authors: Vanessa Sklyarova, Jenya Chelishev, Andreea Dogaru, Igor Medvedev,
Victor Lempitsky, Egor Zakharov
- Abstract summary: This work proposes an approach capable of accurate hair geometry reconstruction at a strand level from a monocular video or multi-view images captured in uncontrolled conditions.
The combined system, named Neural Haircut, achieves high realism and personalization of the reconstructed hairstyles.
- Score: 4.714310894654027
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Generating realistic human 3D reconstructions using image or video data is
essential for various communication and entertainment applications. While
existing methods achieved impressive results for body and facial regions,
realistic hair modeling still remains challenging due to its high mechanical
complexity. This work proposes an approach capable of accurate hair geometry
reconstruction at a strand level from a monocular video or multi-view images
captured in uncontrolled lighting conditions. Our method has two stages, with
the first stage performing joint reconstruction of coarse hair and bust shapes
and hair orientation using implicit volumetric representations. The second
stage then estimates a strand-level hair reconstruction by reconciling in a
single optimization process the coarse volumetric constraints with hair strand
and hairstyle priors learned from the synthetic data. To further increase the
reconstruction fidelity, we incorporate image-based losses into the fitting
process using a new differentiable renderer. The combined system, named Neural
Haircut, achieves high realism and personalization of the reconstructed
hairstyles.
Related papers
- MonoHair: High-Fidelity Hair Modeling from a Monocular Video [40.27026803872373]
MonoHair is a generic framework to achieve high-fidelity hair reconstruction from a monocular video.
Our approach bifurcates the hair modeling process into two main stages: precise exterior reconstruction and interior structure inference.
Our experiments demonstrate that our method exhibits robustness across diverse hairstyles and achieves state-of-the-art performance.
arXiv Detail & Related papers (2024-03-27T08:48:47Z) - GaussianHair: Hair Modeling and Rendering with Light-aware Gaussians [41.52673678183542]
This paper presents GaussianHair, a novel explicit hair representation.
It enables comprehensive modeling of hair geometry and appearance from images, fostering innovative illumination effects and dynamic animation capabilities.
We further enhance this model with the "GaussianHair Scattering Model", adept at recreating the slender structure of hair strands and accurately capturing their local diffuse color in uniform lighting.
arXiv Detail & Related papers (2024-02-16T07:13:24Z) - HAAR: Text-Conditioned Generative Model of 3D Strand-based Human
Hairstyles [85.12672855502517]
We present HAAR, a new strand-based generative model for 3D human hairstyles.
Based on textual inputs, HAAR produces 3D hairstyles that could be used as production-level assets in modern computer graphics engines.
arXiv Detail & Related papers (2023-12-18T19:19:32Z) - ReconFusion: 3D Reconstruction with Diffusion Priors [104.73604630145847]
We present ReconFusion to reconstruct real-world scenes using only a few photos.
Our approach leverages a diffusion prior for novel view synthesis, trained on synthetic and multiview datasets.
Our method synthesizes realistic geometry and texture in underconstrained regions while preserving the appearance of observed regions.
arXiv Detail & Related papers (2023-12-05T18:59:58Z) - Generalizable One-shot Neural Head Avatar [90.50492165284724]
We present a method that reconstructs and animates a 3D head avatar from a single-view portrait image.
We propose a framework that not only generalizes to unseen identities based on a single-view image, but also captures characteristic details within and beyond the face area.
arXiv Detail & Related papers (2023-06-14T22:33:09Z) - HairStep: Transfer Synthetic to Real Using Strand and Depth Maps for
Single-View 3D Hair Modeling [55.57803336895614]
We tackle the challenging problem of learning-based single-view 3D hair modeling.
We first propose a novel intermediate representation, termed as HairStep, which consists of a strand map and a depth map.
It is found that HairStep not only provides sufficient information for accurate 3D hair modeling, but also is feasible to be inferred from real images.
arXiv Detail & Related papers (2023-03-05T15:28:13Z) - Style Your Hair: Latent Optimization for Pose-Invariant Hairstyle
Transfer via Local-Style-Aware Hair Alignment [29.782276472922398]
We propose a pose-invariant hairstyle transfer model equipped with latent optimization and a newly presented local-style-matching loss.
Our model has strengths in transferring a hairstyle under larger pose differences and preserving local hairstyle textures.
arXiv Detail & Related papers (2022-08-16T14:23:54Z) - Neural 3D Reconstruction in the Wild [86.6264706256377]
We introduce a new method that enables efficient and accurate surface reconstruction from Internet photo collections.
We present a new benchmark and protocol for evaluating reconstruction performance on such in-the-wild scenes.
arXiv Detail & Related papers (2022-05-25T17:59:53Z) - HVH: Learning a Hybrid Neural Volumetric Representation for Dynamic Hair
Performance Capture [11.645769995924548]
Capturing and rendering life-like hair is particularly challenging due to its fine geometric structure, the complex physical interaction and its non-trivial visual appearance.
In this paper, we use a novel, volumetric hair representation that is com-posed of thousands of primitives.
Our method can not only create realistic renders of recorded multi-view sequences, but also create renderings for new hair configurations by providing new control signals.
arXiv Detail & Related papers (2021-12-13T18:57:50Z) - Towards High Fidelity Monocular Face Reconstruction with Rich
Reflectance using Self-supervised Learning and Ray Tracing [49.759478460828504]
Methods combining deep neural network encoders with differentiable rendering have opened up the path for very fast monocular reconstruction of geometry, lighting and reflectance.
ray tracing was introduced for monocular face reconstruction within a classic optimization-based framework.
We propose a new method that greatly improves reconstruction quality and robustness in general scenes.
arXiv Detail & Related papers (2021-03-29T08:58:10Z) - Intuitive, Interactive Beard and Hair Synthesis with Generative Models [38.93415643177721]
We present an interactive approach to synthesizing realistic variations in facial hair in images.
We employ a neural network pipeline that synthesizes realistic and detailed images of facial hair directly in the target image in under one second.
We show compelling interactive editing results with a prototype user interface that allows novice users to progressively refine the generated image to match their desired hairstyle.
arXiv Detail & Related papers (2020-04-15T01:20:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.