Towards Unified 3D Hair Reconstruction from Single-View Portraits

• URL: http://arxiv.org/abs/2409.16863v1
• Date: Wed, 25 Sep 2024 12:21:31 GMT
• Title: Towards Unified 3D Hair Reconstruction from Single-View Portraits
• Authors: Yujian Zheng, Yuda Qiu, Leyang Jin, Chongyang Ma, Haibin Huang, Di Zhang, Pengfei Wan, Xiaoguang Han,
• Abstract summary: We propose a novel strategy to enable single-view 3D reconstruction for a variety of hair types via a unified pipeline. Our experiments demonstrate that reconstructing braided and un-braided 3D hair from single-view images via a unified approach is possible.
• Score: 27.404011546957104
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Single-view 3D hair reconstruction is challenging, due to the wide range of shape variations among diverse hairstyles. Current state-of-the-art methods are specialized in recovering un-braided 3D hairs and often take braided styles as their failure cases, because of the inherent difficulty to define priors for complex hairstyles, whether rule-based or data-based. We propose a novel strategy to enable single-view 3D reconstruction for a variety of hair types via a unified pipeline. To achieve this, we first collect a large-scale synthetic multi-view hair dataset SynMvHair with diverse 3D hair in both braided and un-braided styles, and learn two diffusion priors specialized on hair. Then we optimize 3D Gaussian-based hair from the priors with two specially designed modules, i.e. view-wise and pixel-wise Gaussian refinement. Our experiments demonstrate that reconstructing braided and un-braided 3D hair from single-view images via a unified approach is possible and our method achieves the state-of-the-art performance in recovering complex hairstyles. It is worth to mention that our method shows good generalization ability to real images, although it learns hair priors from synthetic data.

Related papers

• Human Hair Reconstruction with Strand-Aligned 3D Gaussians [39.32397354314153]
  We introduce a new hair modeling method that uses a dual representation of classical hair strands and 3D Gaussians. In contrast to recent approaches that leverage unstructured Gaussians to model human avatars, our method reconstructs the hair using 3D polylines, or strands. Our method, named Gaussian Haircut, is evaluated on synthetic and real scenes and demonstrates state-of-the-art performance in the task of strand-based hair reconstruction.
  arXiv  Detail & Related papers  (2024-09-23T07:49:46Z)
• Perm: A Parametric Representation for Multi-Style 3D Hair Modeling [22.790597419351528]
  Perm is a learned parametric model of human 3D hair designed to facilitate various hair-related applications. We propose to disentangle the global hair shape and local strand details using a PCA-based strand representation in the frequency domain. These textures are later parameterized with different generative models, emulating common stages in the hair modeling process.
  arXiv  Detail & Related papers  (2024-07-28T10:05:11Z)
• Style-NeRF2NeRF: 3D Style Transfer From Style-Aligned Multi-View Images [54.56070204172398]
  We propose a simple yet effective pipeline for stylizing a 3D scene. We perform 3D style transfer by refining the source NeRF model using stylized images generated by a style-aligned image-to-image diffusion model. We demonstrate that our method can transfer diverse artistic styles to real-world 3D scenes with competitive quality.
  arXiv  Detail & Related papers  (2024-06-19T09:36:18Z)
• MonoHair: High-Fidelity Hair Modeling from a Monocular Video [40.27026803872373]
  MonoHair is a generic framework to achieve high-fidelity hair reconstruction from a monocular video. Our approach bifurcates the hair modeling process into two main stages: precise exterior reconstruction and interior structure inference. Our experiments demonstrate that our method exhibits robustness across diverse hairstyles and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-03-27T08:48:47Z)
• HAAR: Text-Conditioned Generative Model of 3D Strand-based Human Hairstyles [85.12672855502517]
  We present HAAR, a new strand-based generative model for 3D human hairstyles. Based on textual inputs, HAAR produces 3D hairstyles that could be used as production-level assets in modern computer graphics engines.
  arXiv  Detail & Related papers  (2023-12-18T19:19:32Z)
• Generalizable One-shot Neural Head Avatar [90.50492165284724]
  We present a method that reconstructs and animates a 3D head avatar from a single-view portrait image. We propose a framework that not only generalizes to unseen identities based on a single-view image, but also captures characteristic details within and beyond the face area.
  arXiv  Detail & Related papers  (2023-06-14T22:33:09Z)
• Neural Haircut: Prior-Guided Strand-Based Hair Reconstruction [4.714310894654027]
  This work proposes an approach capable of accurate hair geometry reconstruction at a strand level from a monocular video or multi-view images captured in uncontrolled conditions. The combined system, named Neural Haircut, achieves high realism and personalization of the reconstructed hairstyles.
  arXiv  Detail & Related papers  (2023-06-09T13:08:34Z)
• StyleRF: Zero-shot 3D Style Transfer of Neural Radiance Fields [52.19291190355375]
  StyleRF (Style Radiance Fields) is an innovative 3D style transfer technique. It employs an explicit grid of high-level features to represent 3D scenes, with which high-fidelity geometry can be reliably restored via volume rendering. It transforms the grid features according to the reference style which directly leads to high-quality zero-shot style transfer.
  arXiv  Detail & Related papers  (2023-03-19T08:26:06Z)
• HairStep: Transfer Synthetic to Real Using Strand and Depth Maps for Single-View 3D Hair Modeling [55.57803336895614]
  We tackle the challenging problem of learning-based single-view 3D hair modeling. We first propose a novel intermediate representation, termed as HairStep, which consists of a strand map and a depth map. It is found that HairStep not only provides sufficient information for accurate 3D hair modeling, but also is feasible to be inferred from real images.
  arXiv  Detail & Related papers  (2023-03-05T15:28:13Z)
• i3DMM: Deep Implicit 3D Morphable Model of Human Heads [115.19943330455887]
  We present the first deep implicit 3D morphable model (i3DMM) of full heads. It not only captures identity-specific geometry, texture, and expressions of the frontal face, but also models the entire head, including hair. We show the merits of i3DMM using ablation studies, comparisons to state-of-the-art models, and applications such as semantic head editing and texture transfer.
  arXiv  Detail & Related papers  (2020-11-28T15:01:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.