Learning Disentangled Representation for One-shot Progressive Face Swapping

• URL: http://arxiv.org/abs/2203.12985v1
• Date: Thu, 24 Mar 2022 11:19:04 GMT
• Title: Learning Disentangled Representation for One-shot Progressive Face Swapping
• Authors: Qi Li, Weining Wang, Chengzhong Xu, Zhenan Sun
• Abstract summary: We present a simple yet efficient method named FaceSwapper, for one-shot face swapping based on Generative Adversarial Networks. Our method consists of a disentangled representation module and a semantic-guided fusion module. Our results show that our method achieves state-of-the-art results on benchmark with fewer training samples.
• Score: 65.98684203654908
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although face swapping has attracted much attention in recent years, it remains a challenging problem. The existing methods leverage a large number of data samples to explore the intrinsic properties of face swapping without taking into account the semantic information of face images. Moreover, the representation of the identity information tends to be fixed, leading to suboptimal face swapping. In this paper, we present a simple yet efficient method named FaceSwapper, for one-shot face swapping based on Generative Adversarial Networks. Our method consists of a disentangled representation module and a semantic-guided fusion module. The disentangled representation module is composed of an attribute encoder and an identity encoder, which aims to achieve the disentanglement of the identity and the attribute information. The identity encoder is more flexible and the attribute encoder contains more details of the attributes than its competitors. Benefiting from the disentangled representation, FaceSwapper can swap face images progressively. In addition, semantic information is introduced into the semantic-guided fusion module to control the swapped area and model the pose and expression more accurately. The experimental results show that our method achieves state-of-the-art results on benchmark datasets with fewer training samples. Our code is publicly available at https://github.com/liqi-casia/FaceSwapper.

Related papers

• Infinite-ID: Identity-preserved Personalization via ID-semantics Decoupling Paradigm [31.06269858216316]
  We propose Infinite-ID, an ID-semantics decoupling paradigm for identity-preserved personalization. We introduce an identity-enhanced training, incorporating an additional image cross-attention module to capture sufficient ID information. We also introduce a feature interaction mechanism that combines a mixed attention module with an AdaIN-mean operation to seamlessly merge the two streams.
  arXiv  Detail & Related papers  (2024-03-18T13:39:53Z)
• Personalized Face Inpainting with Diffusion Models by Parallel Visual Attention [55.33017432880408]
  This paper proposes the use of Parallel Visual Attention (PVA) in conjunction with diffusion models to improve inpainting results. We train the added attention modules and identity encoder on CelebAHQ-IDI, a dataset proposed for identity-preserving face inpainting. Experiments demonstrate that PVA attains unparalleled identity resemblance in both face inpainting and face inpainting with language guidance tasks.
  arXiv  Detail & Related papers  (2023-12-06T15:39:03Z)
• When StyleGAN Meets Stable Diffusion: a $\mathscr{W}_+$ Adapter for Personalized Image Generation [60.305112612629465]
  Text-to-image diffusion models have excelled in producing diverse, high-quality, and photo-realistic images. We present a novel use of the extended StyleGAN embedding space $mathcalW_+$ to achieve enhanced identity preservation and disentanglement for diffusion models. Our method adeptly generates personalized text-to-image outputs that are not only compatible with prompt descriptions but also amenable to common StyleGAN editing directions.
  arXiv  Detail & Related papers  (2023-11-29T09:05:14Z)
• BlendFace: Re-designing Identity Encoders for Face-Swapping [2.320417845168326]
  BlendFace is a novel identity encoder for face-swapping. It disentangles identity features into generators and guides generators properly as an identity loss function. Extensive experiments demonstrate that BlendFace improves the identity-attribute disentanglement in face-swapping models.
  arXiv  Detail & Related papers  (2023-07-20T13:17:30Z)
• FaceDancer: Pose- and Occlusion-Aware High Fidelity Face Swapping [62.38898610210771]
  We present a new single-stage method for subject face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR)
  arXiv  Detail & Related papers  (2022-10-19T11:31:38Z)
• ShapeEditer: a StyleGAN Encoder for Face Swapping [6.848723869850855]
  We propose a novel encoder, called ShapeEditor, for high-resolution, realistic and high-fidelity face exchange. Our key idea is to use an advanced pretrained high-quality random face image generator, i.e. StyleGAN, as backbone. For learning to map into the latent space of StyleGAN, we propose a set of self-supervised loss functions.
  arXiv  Detail & Related papers  (2021-06-26T09:38:45Z)
• FaceController: Controllable Attribute Editing for Face in the Wild [74.56117807309576]
  We propose a simple feed-forward network to generate high-fidelity manipulated faces. By simply employing some existing and easy-obtainable prior information, our method can control, transfer, and edit diverse attributes of faces in the wild. In our method, we decouple identity, expression, pose, and illumination using 3D priors; separate texture and colors by using region-wise style codes.
  arXiv  Detail & Related papers  (2021-02-23T02:47:28Z)
• Fine-grained Image-to-Image Transformation towards Visual Recognition [102.51124181873101]
  We aim at transforming an image with a fine-grained category to synthesize new images that preserve the identity of the input image. We adopt a model based on generative adversarial networks to disentangle the identity related and unrelated factors of an image. Experiments on the CompCars and Multi-PIE datasets demonstrate that our model preserves the identity of the generated images much better than the state-of-the-art image-to-image transformation models.
  arXiv  Detail & Related papers  (2020-01-12T05:26:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.