Stable-Hair: Real-World Hair Transfer via Diffusion Model

• URL: http://arxiv.org/abs/2407.14078v2
• Date: Tue, 10 Dec 2024 16:04:50 GMT
• Title: Stable-Hair: Real-World Hair Transfer via Diffusion Model
• Authors: Yuxuan Zhang, Qing Zhang, Yiren Song, Jichao Zhang, Hao Tang, Jiaming Liu,
• Abstract summary: Current hair transfer methods struggle to handle diverse and intricate hairstyles, limiting their applicability in real-world scenarios. We propose a novel diffusion-based hair transfer framework, named textitStable-Hair, which robustly transfers a wide range of real-world hairstyles to user-provided faces for virtual hair try-on.
• Score: 26.880396643803998
• License:
• Abstract: Current hair transfer methods struggle to handle diverse and intricate hairstyles, limiting their applicability in real-world scenarios. In this paper, we propose a novel diffusion-based hair transfer framework, named \textit{Stable-Hair}, which robustly transfers a wide range of real-world hairstyles to user-provided faces for virtual hair try-on. To achieve this goal, our Stable-Hair framework is designed as a two-stage pipeline. In the first stage, we train a Bald Converter alongside stable diffusion to remove hair from the user-provided face images, resulting in bald images. In the second stage, we specifically designed a Hair Extractor and a Latent IdentityNet to transfer the target hairstyle with highly detailed and high-fidelity to the bald image. The Hair Extractor is trained to encode reference images with the desired hairstyles, while the Latent IdentityNet ensures consistency in identity and background. To minimize color deviations between source images and transfer results, we introduce a novel Latent ControlNet architecture, which functions as both the Bald Converter and Latent IdentityNet. After training on our curated triplet dataset, our method accurately transfers highly detailed and high-fidelity hairstyles to the source images. Extensive experiments demonstrate that our approach achieves state-of-the-art performance compared to existing hair transfer methods. Project page: \textcolor{red}{\url{https://xiaojiu-z.github.io/Stable-Hair.github.io/}}

Related papers

• What to Preserve and What to Transfer: Faithful, Identity-Preserving Diffusion-based Hairstyle Transfer [35.80645300182437]
  Existing hairstyle transfer approaches rely on StyleGAN. We propose a one-stage hairstyle transfer diffusion model, HairFusion, that applies to real-world scenarios. Our method achieves state-of-the-art performance compared to the existing methods in preserving the integrity of both the transferred hairstyle and the surrounding features.
  arXiv  Detail & Related papers  (2024-08-29T11:30:21Z)
• HairFastGAN: Realistic and Robust Hair Transfer with a Fast Encoder-Based Approach [3.737361598712633]
  We present the HairFast model, which achieves high resolution, near real-time performance, and superior reconstruction. Our solution includes a new architecture operating in the FS latent space of StyleGAN. In the most difficult scenario of transferring both shape and color of a hairstyle from different images, our method performs in less than a second on the Nvidia V100.
  arXiv  Detail & Related papers  (2024-04-01T12:59:49Z)
• Improving Diffusion Models for Authentic Virtual Try-on in the Wild [53.96244595495942]
  This paper considers image-based virtual try-on, which renders an image of a person wearing a curated garment. We propose a novel diffusion model that improves garment fidelity and generates authentic virtual try-on images. We present a customization method using a pair of person-garment images, which significantly improves fidelity and authenticity.
  arXiv  Detail & Related papers  (2024-03-08T08:12:18Z)
• HAAR: Text-Conditioned Generative Model of 3D Strand-based Human Hairstyles [85.12672855502517]
  We present HAAR, a new strand-based generative model for 3D human hairstyles. Based on textual inputs, HAAR produces 3D hairstyles that could be used as production-level assets in modern computer graphics engines.
  arXiv  Detail & Related papers  (2023-12-18T19:19:32Z)
• Text-Guided Generation and Editing of Compositional 3D Avatars [59.584042376006316]
  Our goal is to create a realistic 3D facial avatar with hair and accessories using only a text description. Existing methods either lack realism, produce unrealistic shapes, or do not support editing.
  arXiv  Detail & Related papers  (2023-09-13T17:59:56Z)
• StyleGAN Salon: Multi-View Latent Optimization for Pose-Invariant Hairstyle Transfer [8.712040236361926]
  The paper seeks to transfer the hairstyle of a reference image to an input photo for virtual hair try-on. We propose a multi-view optimization framework that uses "two different views" of reference composites to semantically guide occluded or ambiguous regions. Our framework produces high-quality results and outperforms prior work in a user study that consists of significantly more challenging hair transfer scenarios.
  arXiv  Detail & Related papers  (2023-04-05T20:49:55Z)
• HairStep: Transfer Synthetic to Real Using Strand and Depth Maps for Single-View 3D Hair Modeling [55.57803336895614]
  We tackle the challenging problem of learning-based single-view 3D hair modeling. We first propose a novel intermediate representation, termed as HairStep, which consists of a strand map and a depth map. It is found that HairStep not only provides sufficient information for accurate 3D hair modeling, but also is feasible to be inferred from real images.
  arXiv  Detail & Related papers  (2023-03-05T15:28:13Z)
• Style Your Hair: Latent Optimization for Pose-Invariant Hairstyle Transfer via Local-Style-Aware Hair Alignment [29.782276472922398]
  We propose a pose-invariant hairstyle transfer model equipped with latent optimization and a newly presented local-style-matching loss. Our model has strengths in transferring a hairstyle under larger pose differences and preserving local hairstyle textures.
  arXiv  Detail & Related papers  (2022-08-16T14:23:54Z)
• HairFIT: Pose-Invariant Hairstyle Transfer via Flow-based Hair Alignment and Semantic-Region-Aware Inpainting [26.688276902813495]
  We propose a novel framework for pose-invariant hairstyle transfer, HairFIT. Our model consists of two stages: 1) flow-based hair alignment and 2) hair synthesis. Our SIM estimator divides the occluded regions in the source image into different semantic regions to reflect their distinct features during the inpainting.
  arXiv  Detail & Related papers  (2022-06-17T06:55:20Z)
• Progressive and Aligned Pose Attention Transfer for Person Image Generation [59.87492938953545]
  This paper proposes a new generative adversarial network for pose transfer, i.e., transferring the pose of a given person to a target pose. We use two types of blocks, namely Pose-Attentional Transfer Block (PATB) and Aligned Pose-Attentional Transfer Bloc (APATB) We verify the efficacy of the model on the Market-1501 and DeepFashion datasets, using quantitative and qualitative measures.
  arXiv  Detail & Related papers  (2021-03-22T07:24:57Z)
• MichiGAN: Multi-Input-Conditioned Hair Image Generation for Portrait Editing [122.82964863607938]
  MichiGAN is a novel conditional image generation method for interactive portrait hair manipulation. We provide user control over every major hair visual factor, including shape, structure, appearance, and background. We also build an interactive portrait hair editing system that enables straightforward manipulation of hair by projecting intuitive and high-level user inputs.
  arXiv  Detail & Related papers  (2020-10-30T17:59:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.