Towards Unified 3D Hair Reconstruction from Single-View Portraits

• URL: http://arxiv.org/abs/2409.16863v1
• Date: Wed, 25 Sep 2024 12:21:31 GMT
• Title: Towards Unified 3D Hair Reconstruction from Single-View Portraits
• Authors: Yujian Zheng, Yuda Qiu, Leyang Jin, Chongyang Ma, Haibin Huang, Di Zhang, Pengfei Wan, Xiaoguang Han,
• Abstract summary: We propose a novel strategy to enable single-view 3D reconstruction for a variety of hair types via a unified pipeline. Our experiments demonstrate that reconstructing braided and un-braided 3D hair from single-view images via a unified approach is possible.
• Score: 27.404011546957104
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Single-view 3D hair reconstruction is challenging, due to the wide range of shape variations among diverse hairstyles. Current state-of-the-art methods are specialized in recovering un-braided 3D hairs and often take braided styles as their failure cases, because of the inherent difficulty to define priors for complex hairstyles, whether rule-based or data-based. We propose a novel strategy to enable single-view 3D reconstruction for a variety of hair types via a unified pipeline. To achieve this, we first collect a large-scale synthetic multi-view hair dataset SynMvHair with diverse 3D hair in both braided and un-braided styles, and learn two diffusion priors specialized on hair. Then we optimize 3D Gaussian-based hair from the priors with two specially designed modules, i.e. view-wise and pixel-wise Gaussian refinement. Our experiments demonstrate that reconstructing braided and un-braided 3D hair from single-view images via a unified approach is possible and our method achieves the state-of-the-art performance in recovering complex hairstyles. It is worth to mention that our method shows good generalization ability to real images, although it learns hair priors from synthetic data.

Related papers

• TANGLED: Generating 3D Hair Strands from Images with Arbitrary Styles and Viewpoints [38.95048174663582]
  Existing text or image-guided generation methods fail to handle the richness and complexity of diverse styles. We present TANGLED, a novel approach for 3D hair strand generation that accommodates diverse image inputs across styles, viewpoints, and quantities of input views.
  arXiv  Detail & Related papers  (2025-02-10T12:26:02Z)
• Arc2Avatar: Generating Expressive 3D Avatars from a Single Image via ID Guidance [69.9745497000557]
  We introduce Arc2Avatar, the first SDS-based method utilizing a human face foundation model as guidance with just a single image as input. Our avatars maintain a dense correspondence with a human face mesh template, allowing blendshape-based expression generation.
  arXiv  Detail & Related papers  (2025-01-09T17:04:33Z)
• FaceLift: Single Image to 3D Head with View Generation and GS-LRM [54.24070918942727]
  FaceLift is a feed-forward approach for rapid, high-quality, 360-degree head reconstruction from a single image. We show that FaceLift outperforms state-of-the-art methods in 3D head reconstruction, highlighting its practical applicability and robust performance on real-world images.
  arXiv  Detail & Related papers  (2024-12-23T18:59:49Z)
• Human Hair Reconstruction with Strand-Aligned 3D Gaussians [39.32397354314153]
  We introduce a new hair modeling method that uses a dual representation of classical hair strands and 3D Gaussians. In contrast to recent approaches that leverage unstructured Gaussians to model human avatars, our method reconstructs the hair using 3D polylines, or strands. Our method, named Gaussian Haircut, is evaluated on synthetic and real scenes and demonstrates state-of-the-art performance in the task of strand-based hair reconstruction.
  arXiv  Detail & Related papers  (2024-09-23T07:49:46Z)
• Perm: A Parametric Representation for Multi-Style 3D Hair Modeling [22.790597419351528]
  Perm is a learned parametric representation of human 3D hair designed to facilitate various hair-related applications. We leverage our strand representation to fit and decompose hair geometry textures into low- to high-frequency hair structures.
  arXiv  Detail & Related papers  (2024-07-28T10:05:11Z)
• Style-NeRF2NeRF: 3D Style Transfer From Style-Aligned Multi-View Images [54.56070204172398]
  We propose a simple yet effective pipeline for stylizing a 3D scene. We perform 3D style transfer by refining the source NeRF model using stylized images generated by a style-aligned image-to-image diffusion model. We demonstrate that our method can transfer diverse artistic styles to real-world 3D scenes with competitive quality.
  arXiv  Detail & Related papers  (2024-06-19T09:36:18Z)
• MonoHair: High-Fidelity Hair Modeling from a Monocular Video [40.27026803872373]
  MonoHair is a generic framework to achieve high-fidelity hair reconstruction from a monocular video. Our approach bifurcates the hair modeling process into two main stages: precise exterior reconstruction and interior structure inference. Our experiments demonstrate that our method exhibits robustness across diverse hairstyles and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-03-27T08:48:47Z)
• HAAR: Text-Conditioned Generative Model of 3D Strand-based Human Hairstyles [85.12672855502517]
  We present HAAR, a new strand-based generative model for 3D human hairstyles. Based on textual inputs, HAAR produces 3D hairstyles that could be used as production-level assets in modern computer graphics engines.
  arXiv  Detail & Related papers  (2023-12-18T19:19:32Z)
• Generalizable One-shot Neural Head Avatar [90.50492165284724]
  We present a method that reconstructs and animates a 3D head avatar from a single-view portrait image. We propose a framework that not only generalizes to unseen identities based on a single-view image, but also captures characteristic details within and beyond the face area.
  arXiv  Detail & Related papers  (2023-06-14T22:33:09Z)
• Neural Haircut: Prior-Guided Strand-Based Hair Reconstruction [4.714310894654027]
  This work proposes an approach capable of accurate hair geometry reconstruction at a strand level from a monocular video or multi-view images captured in uncontrolled conditions. The combined system, named Neural Haircut, achieves high realism and personalization of the reconstructed hairstyles.
  arXiv  Detail & Related papers  (2023-06-09T13:08:34Z)
• HairStep: Transfer Synthetic to Real Using Strand and Depth Maps for Single-View 3D Hair Modeling [55.57803336895614]
  We tackle the challenging problem of learning-based single-view 3D hair modeling. We first propose a novel intermediate representation, termed as HairStep, which consists of a strand map and a depth map. It is found that HairStep not only provides sufficient information for accurate 3D hair modeling, but also is feasible to be inferred from real images.
  arXiv  Detail & Related papers  (2023-03-05T15:28:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.