MichiGAN: Multi-Input-Conditioned Hair Image Generation for Portrait Editing

• URL: http://arxiv.org/abs/2010.16417v1
• Date: Fri, 30 Oct 2020 17:59:10 GMT
• Title: MichiGAN: Multi-Input-Conditioned Hair Image Generation for Portrait Editing
• Authors: Zhentao Tan and Menglei Chai and Dongdong Chen and Jing Liao and Qi Chu and Lu Yuan and Sergey Tulyakov and Nenghai Yu
• Abstract summary: MichiGAN is a novel conditional image generation method for interactive portrait hair manipulation. We provide user control over every major hair visual factor, including shape, structure, appearance, and background. We also build an interactive portrait hair editing system that enables straightforward manipulation of hair by projecting intuitive and high-level user inputs.
• Score: 122.82964863607938
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the recent success of face image generation with GANs, conditional hair editing remains challenging due to the under-explored complexity of its geometry and appearance. In this paper, we present MichiGAN (Multi-Input-Conditioned Hair Image GAN), a novel conditional image generation method for interactive portrait hair manipulation. To provide user control over every major hair visual factor, we explicitly disentangle hair into four orthogonal attributes, including shape, structure, appearance, and background. For each of them, we design a corresponding condition module to represent, process, and convert user inputs, and modulate the image generation pipeline in ways that respect the natures of different visual attributes. All these condition modules are integrated with the backbone generator to form the final end-to-end network, which allows fully-conditioned hair generation from multiple user inputs. Upon it, we also build an interactive portrait hair editing system that enables straightforward manipulation of hair by projecting intuitive and high-level user inputs such as painted masks, guiding strokes, or reference photos to well-defined condition representations. Through extensive experiments and evaluations, we demonstrate the superiority of our method regarding both result quality and user controllability. The code is available at https://github.com/tzt101/MichiGAN.

Related papers

• PixWizard: Versatile Image-to-Image Visual Assistant with Open-Language Instructions [66.92809850624118]
  PixWizard is an image-to-image visual assistant designed for image generation, manipulation, and translation based on free-from language instructions. We tackle a variety of vision tasks into a unified image-text-to-image generation framework and curate an Omni Pixel-to-Pixel Instruction-Tuning dataset. Our experiments demonstrate that PixWizard not only shows impressive generative and understanding abilities for images with diverse resolutions but also exhibits promising generalization capabilities with unseen tasks and human instructions.
  arXiv  Detail & Related papers  (2024-09-23T17:59:46Z)
• Perm: A Parametric Representation for Multi-Style 3D Hair Modeling [22.790597419351528]
  Perm is a learned parametric model of human 3D hair designed to facilitate various hair-related applications. We propose to disentangle the global hair shape and local strand details using a PCA-based strand representation in the frequency domain. These textures are later parameterized with different generative models, emulating common stages in the hair modeling process.
  arXiv  Detail & Related papers  (2024-07-28T10:05:11Z)
• MeGA: Hybrid Mesh-Gaussian Head Avatar for High-Fidelity Rendering and Head Editing [34.31657241047574]
  We propose a Hybrid Mesh-Gaussian Head Avatar (MeGA) that models different head components with more suitable representations. MeGA generates higher-fidelity renderings for the whole head and naturally supports more downstream tasks. Experiments on the NeRSemble dataset demonstrate the effectiveness of our designs.
  arXiv  Detail & Related papers  (2024-04-29T18:10:12Z)
• HairCLIPv2: Unifying Hair Editing via Proxy Feature Blending [94.39417893972262]
  HairCLIP is the first work that enables hair editing based on text descriptions or reference images. In this paper, we propose HairCLIPv2, aiming to support all the aforementioned interactions with one unified framework. The key idea is to convert all the hair editing tasks into hair transfer tasks, with editing conditions converted into different proxies accordingly.
  arXiv  Detail & Related papers  (2023-10-16T17:59:58Z)
• Generalizable One-shot Neural Head Avatar [90.50492165284724]
  We present a method that reconstructs and animates a 3D head avatar from a single-view portrait image. We propose a framework that not only generalizes to unseen identities based on a single-view image, but also captures characteristic details within and beyond the face area.
  arXiv  Detail & Related papers  (2023-06-14T22:33:09Z)
• HairCLIP: Design Your Hair by Text and Reference Image [100.85116679883724]
  This paper proposes a new hair editing interaction mode, which enables manipulating hair attributes individually or jointly. We encode the image and text conditions in a shared embedding space and propose a unified hair editing framework. With the carefully designed network structures and loss functions, our framework can perform high-quality hair editing.
  arXiv  Detail & Related papers  (2021-12-09T18:59:58Z)
• Generating Person Images with Appearance-aware Pose Stylizer [66.44220388377596]
  We present a novel end-to-end framework to generate realistic person images based on given person poses and appearances. The core of our framework is a novel generator called Appearance-aware Pose Stylizer (APS) which generates human images by coupling the target pose with the conditioned person appearance progressively.
  arXiv  Detail & Related papers  (2020-07-17T15:58:05Z)
• Intuitive, Interactive Beard and Hair Synthesis with Generative Models [38.93415643177721]
  We present an interactive approach to synthesizing realistic variations in facial hair in images. We employ a neural network pipeline that synthesizes realistic and detailed images of facial hair directly in the target image in under one second. We show compelling interactive editing results with a prototype user interface that allows novice users to progressively refine the generated image to match their desired hairstyle.
  arXiv  Detail & Related papers  (2020-04-15T01:20:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.