Neural Haircut: Prior-Guided Strand-Based Hair Reconstruction

• URL: http://arxiv.org/abs/2306.05872v2
• Date: Mon, 12 Jun 2023 10:31:38 GMT
• Title: Neural Haircut: Prior-Guided Strand-Based Hair Reconstruction
• Authors: Vanessa Sklyarova, Jenya Chelishev, Andreea Dogaru, Igor Medvedev, Victor Lempitsky, Egor Zakharov
• Abstract summary: This work proposes an approach capable of accurate hair geometry reconstruction at a strand level from a monocular video or multi-view images captured in uncontrolled conditions. The combined system, named Neural Haircut, achieves high realism and personalization of the reconstructed hairstyles.
• Score: 4.714310894654027
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Generating realistic human 3D reconstructions using image or video data is essential for various communication and entertainment applications. While existing methods achieved impressive results for body and facial regions, realistic hair modeling still remains challenging due to its high mechanical complexity. This work proposes an approach capable of accurate hair geometry reconstruction at a strand level from a monocular video or multi-view images captured in uncontrolled lighting conditions. Our method has two stages, with the first stage performing joint reconstruction of coarse hair and bust shapes and hair orientation using implicit volumetric representations. The second stage then estimates a strand-level hair reconstruction by reconciling in a single optimization process the coarse volumetric constraints with hair strand and hairstyle priors learned from the synthetic data. To further increase the reconstruction fidelity, we incorporate image-based losses into the fitting process using a new differentiable renderer. The combined system, named Neural Haircut, achieves high realism and personalization of the reconstructed hairstyles.

Related papers

• Towards Unified 3D Hair Reconstruction from Single-View Portraits [27.404011546957104]
  We propose a novel strategy to enable single-view 3D reconstruction for a variety of hair types via a unified pipeline. Our experiments demonstrate that reconstructing braided and un-braided 3D hair from single-view images via a unified approach is possible.
  arXiv  Detail & Related papers  (2024-09-25T12:21:31Z)
• Human Hair Reconstruction with Strand-Aligned 3D Gaussians [39.32397354314153]
  We introduce a new hair modeling method that uses a dual representation of classical hair strands and 3D Gaussians. In contrast to recent approaches that leverage unstructured Gaussians to model human avatars, our method reconstructs the hair using 3D polylines, or strands. Our method, named Gaussian Haircut, is evaluated on synthetic and real scenes and demonstrates state-of-the-art performance in the task of strand-based hair reconstruction.
  arXiv  Detail & Related papers  (2024-09-23T07:49:46Z)
• Perm: A Parametric Representation for Multi-Style 3D Hair Modeling [22.790597419351528]
  Perm is a learned parametric model of human 3D hair designed to facilitate various hair-related applications. We propose to disentangle the global hair shape and local strand details using a PCA-based strand representation in the frequency domain. These textures are later parameterized with different generative models, emulating common stages in the hair modeling process.
  arXiv  Detail & Related papers  (2024-07-28T10:05:11Z)
• MonoHair: High-Fidelity Hair Modeling from a Monocular Video [40.27026803872373]
  MonoHair is a generic framework to achieve high-fidelity hair reconstruction from a monocular video. Our approach bifurcates the hair modeling process into two main stages: precise exterior reconstruction and interior structure inference. Our experiments demonstrate that our method exhibits robustness across diverse hairstyles and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-03-27T08:48:47Z)
• GaussianHair: Hair Modeling and Rendering with Light-aware Gaussians [41.52673678183542]
  This paper presents GaussianHair, a novel explicit hair representation. It enables comprehensive modeling of hair geometry and appearance from images, fostering innovative illumination effects and dynamic animation capabilities. We further enhance this model with the "GaussianHair Scattering Model", adept at recreating the slender structure of hair strands and accurately capturing their local diffuse color in uniform lighting.
  arXiv  Detail & Related papers  (2024-02-16T07:13:24Z)
• HAAR: Text-Conditioned Generative Model of 3D Strand-based Human Hairstyles [85.12672855502517]
  We present HAAR, a new strand-based generative model for 3D human hairstyles. Based on textual inputs, HAAR produces 3D hairstyles that could be used as production-level assets in modern computer graphics engines.
  arXiv  Detail & Related papers  (2023-12-18T19:19:32Z)
• ReconFusion: 3D Reconstruction with Diffusion Priors [104.73604630145847]
  We present ReconFusion to reconstruct real-world scenes using only a few photos. Our approach leverages a diffusion prior for novel view synthesis, trained on synthetic and multiview datasets. Our method synthesizes realistic geometry and texture in underconstrained regions while preserving the appearance of observed regions.
  arXiv  Detail & Related papers  (2023-12-05T18:59:58Z)
• Generalizable One-shot Neural Head Avatar [90.50492165284724]
  We present a method that reconstructs and animates a 3D head avatar from a single-view portrait image. We propose a framework that not only generalizes to unseen identities based on a single-view image, but also captures characteristic details within and beyond the face area.
  arXiv  Detail & Related papers  (2023-06-14T22:33:09Z)
• HairStep: Transfer Synthetic to Real Using Strand and Depth Maps for Single-View 3D Hair Modeling [55.57803336895614]
  We tackle the challenging problem of learning-based single-view 3D hair modeling. We first propose a novel intermediate representation, termed as HairStep, which consists of a strand map and a depth map. It is found that HairStep not only provides sufficient information for accurate 3D hair modeling, but also is feasible to be inferred from real images.
  arXiv  Detail & Related papers  (2023-03-05T15:28:13Z)
• Neural 3D Reconstruction in the Wild [86.6264706256377]
  We introduce a new method that enables efficient and accurate surface reconstruction from Internet photo collections. We present a new benchmark and protocol for evaluating reconstruction performance on such in-the-wild scenes.
  arXiv  Detail & Related papers  (2022-05-25T17:59:53Z)
• Towards High Fidelity Monocular Face Reconstruction with Rich Reflectance using Self-supervised Learning and Ray Tracing [49.759478460828504]
  Methods combining deep neural network encoders with differentiable rendering have opened up the path for very fast monocular reconstruction of geometry, lighting and reflectance. ray tracing was introduced for monocular face reconstruction within a classic optimization-based framework. We propose a new method that greatly improves reconstruction quality and robustness in general scenes.
  arXiv  Detail & Related papers  (2021-03-29T08:58:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.