PSGAN++: Robust Detail-Preserving Makeup Transfer and Removal

• URL: http://arxiv.org/abs/2105.12324v1
• Date: Wed, 26 May 2021 04:37:57 GMT
• Title: PSGAN++: Robust Detail-Preserving Makeup Transfer and Removal
• Authors: Si Liu, Wentao Jiang, Chen Gao, Ran He, Jiashi Feng, Bo Li, Shuicheng Yan
• Abstract summary: PSGAN++ is capable of performing both detail-preserving makeup transfer and effective makeup removal. For makeup transfer, PSGAN++ uses a Makeup Distill Network to extract makeup information. For makeup removal, PSGAN++ applies an Identity Distill Network to embed the identity information from with-makeup images into identity matrices.
• Score: 176.47249346856393
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we address the makeup transfer and removal tasks simultaneously, which aim to transfer the makeup from a reference image to a source image and remove the makeup from the with-makeup image respectively. Existing methods have achieved much advancement in constrained scenarios, but it is still very challenging for them to transfer makeup between images with large pose and expression differences, or handle makeup details like blush on cheeks or highlight on the nose. In addition, they are hardly able to control the degree of makeup during transferring or to transfer a specified part in the input face. In this work, we propose the PSGAN++, which is capable of performing both detail-preserving makeup transfer and effective makeup removal. For makeup transfer, PSGAN++ uses a Makeup Distill Network to extract makeup information, which is embedded into spatial-aware makeup matrices. We also devise an Attentive Makeup Morphing module that specifies how the makeup in the source image is morphed from the reference image, and a makeup detail loss to supervise the model within the selected makeup detail area. On the other hand, for makeup removal, PSGAN++ applies an Identity Distill Network to embed the identity information from with-makeup images into identity matrices. Finally, the obtained makeup/identity matrices are fed to a Style Transfer Network that is able to edit the feature maps to achieve makeup transfer or removal. To evaluate the effectiveness of our PSGAN++, we collect a Makeup Transfer In the Wild dataset that contains images with diverse poses and expressions and a Makeup Transfer High-Resolution dataset that contains high-resolution images. Experiments demonstrate that PSGAN++ not only achieves state-of-the-art results with fine makeup details even in cases of large pose/expression differences but also can perform partial or degree-controllable makeup transfer.

Related papers

• DiffAM: Diffusion-based Adversarial Makeup Transfer for Facial Privacy Protection [60.73609509756533]
  DiffAM is a novel approach to generate high-quality protected face images with adversarial makeup transferred from reference images. Experiments demonstrate that DiffAM achieves higher visual quality and attack success rates with a gain of 12.98% under black-box setting.
  arXiv  Detail & Related papers  (2024-05-16T08:05:36Z)
• Stable-Makeup: When Real-World Makeup Transfer Meets Diffusion Model [35.01727715493926]
  Current makeup transfer methods are limited to simple makeup styles, making them difficult to apply in real-world scenarios. We introduce Stable-Makeup, a novel diffusion-based makeup transfer method capable of robustly transferring a wide range of real-world makeup.
  arXiv  Detail & Related papers  (2024-03-12T15:53:14Z)
• BeautyREC: Robust, Efficient, and Content-preserving Makeup Transfer [73.39598356799974]
  We propose a Robust, Efficient, and Component-specific makeup transfer method (abbreviated as BeautyREC) A component-specific correspondence to directly transfer the makeup style of a reference image to the corresponding components. As an auxiliary, the long-range visual dependencies of Transformer are introduced for effective global makeup transfer.
  arXiv  Detail & Related papers  (2022-12-12T12:38:27Z)
• SOGAN: 3D-Aware Shadow and Occlusion Robust GAN for Makeup Transfer [68.38955698584758]
  We propose a novel makeup transfer method called 3D-Aware Shadow and Occlusion Robust GAN (SOGAN) We first fit a 3D face model and then disentangle the faces into shape and texture. In the texture branch, we map the texture to the UV space and design a UV texture generator to transfer the makeup.
  arXiv  Detail & Related papers  (2021-04-21T14:48:49Z)
• Facial Attribute Transformers for Precise and Robust Makeup Transfer [79.41060385695977]
  We propose a novel Facial Attribute Transformer (FAT) and its variant Spatial FAT for high-quality makeup transfer. FAT is able to model the semantic correspondences and interactions between the source face and reference face, and then precisely estimate and transfer the facial attributes. We also integrate thin plate splines (TPS) into FAT, thus creating Spatial FAT, which is the first method that can transfer geometric attributes in addition to color and texture.
  arXiv  Detail & Related papers  (2021-04-07T03:39:02Z)
• Lipstick ain't enough: Beyond Color Matching for In-the-Wild Makeup Transfer [20.782984081934213]
  We propose a holistic makeup transfer framework that can handle all the mentioned makeup components. It consists of an improved color transfer branch and a novel pattern transfer branch to learn all makeup properties. Our framework achieves the state of the art performance on both light and extreme makeup styles.
  arXiv  Detail & Related papers  (2021-04-05T12:12:56Z)
• MakeupBag: Disentangling Makeup Extraction and Application [0.0]
  MakeupBag is a novel method for automatic makeup style transfer. It allows customization and pixel specific modification of the extracted makeup style. In a comparative analysis, MakeupBag is shown to outperform current state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-12-03T18:44:24Z)
• Cosmetic-Aware Makeup Cleanser [109.41917954315784]
  Face verification aims at determining whether a pair of face images belongs to the same identity. Recent studies have revealed the negative impact of facial makeup on the verification performance. This paper proposes a semanticaware makeup cleanser (SAMC) to remove facial makeup under different poses and expressions.
  arXiv  Detail & Related papers  (2020-04-20T09:18:23Z)
• Local Facial Makeup Transfer via Disentangled Representation [18.326829657548025]
  We propose a novel unified adversarial disentangling network to decompose face images into four independent components, i.e., personal identity, lips makeup style, eyes makeup style and face makeup style. Our approach can produce more realistic and accurate makeup transfer results compared to the state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-03-27T00:25:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.