Face Transformer: Towards High Fidelity and Accurate Face Swapping
- URL: http://arxiv.org/abs/2304.02530v1
- Date: Wed, 5 Apr 2023 15:51:44 GMT
- Title: Face Transformer: Towards High Fidelity and Accurate Face Swapping
- Authors: Kaiwen Cui, Rongliang Wu, Fangneng Zhan, Shijian Lu
- Abstract summary: Face swapping aims to generate swapped images that fuse the identity of source faces and the attributes of target faces.
This paper presents Face Transformer, a novel face swapping network that can accurately preserve source identities and target attributes simultaneously.
- Score: 54.737909435708936
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Face swapping aims to generate swapped images that fuse the identity of
source faces and the attributes of target faces. Most existing works address
this challenging task through 3D modelling or generation using generative
adversarial networks (GANs), but 3D modelling suffers from limited
reconstruction accuracy and GANs often struggle in preserving subtle yet
important identity details of source faces (e.g., skin colors, face features)
and structural attributes of target faces (e.g., face shapes, facial
expressions). This paper presents Face Transformer, a novel face swapping
network that can accurately preserve source identities and target attributes
simultaneously in the swapped face images. We introduce a transformer network
for the face swapping task, which learns high-quality semantic-aware
correspondence between source and target faces and maps identity features of
source faces to the corresponding region in target faces. The high-quality
semantic-aware correspondence enables smooth and accurate transfer of source
identity information with minimal modification of target shapes and
expressions. In addition, our Face Transformer incorporates a multi-scale
transformation mechanism for preserving the rich fine facial details. Extensive
experiments show that our Face Transformer achieves superior face swapping
performance qualitatively and quantitatively.
Related papers
- G2Face: High-Fidelity Reversible Face Anonymization via Generative and Geometric Priors [71.69161292330504]
Reversible face anonymization seeks to replace sensitive identity information in facial images with synthesized alternatives.
This paper introduces Gtextsuperscript2Face, which leverages both generative and geometric priors to enhance identity manipulation.
Our method outperforms existing state-of-the-art techniques in face anonymization and recovery, while preserving high data utility.
arXiv Detail & Related papers (2024-08-18T12:36:47Z) - High-Fidelity Face Swapping with Style Blending [16.024260677867076]
We propose an innovative end-to-end framework for high-fidelity face swapping.
First, we introduce a StyleGAN-based facial attributes encoder that extracts essential features from faces and inverts them into a latent style code.
Second, we introduce an attention-based style blending module to effectively transfer Face IDs from source to target.
arXiv Detail & Related papers (2023-12-17T23:22:37Z) - FlowFace: Semantic Flow-guided Shape-aware Face Swapping [43.166181219154936]
We propose a semantic flow-guided two-stage framework for shape-aware face swapping, namely FlowFace.
Our FlowFace consists of a face reshaping network and a face swapping network.
We employ a pre-trained face masked autoencoder to extract facial features from both the source face and the target face.
arXiv Detail & Related papers (2022-12-06T07:23:39Z) - FaceFormer: Scale-aware Blind Face Restoration with Transformers [18.514630131883536]
We propose a novel scale-aware blind face restoration framework, named FaceFormer, which formulates facial feature restoration as scale-aware transformation.
Our proposed method trained with synthetic dataset generalizes better to a natural low quality images than current state-of-the-arts.
arXiv Detail & Related papers (2022-07-20T10:08:34Z) - FaceTuneGAN: Face Autoencoder for Convolutional Expression Transfer
Using Neural Generative Adversarial Networks [0.7043489166804575]
We present FaceTuneGAN, a new 3D face model representation decomposing and encoding separately facial identity and facial expression.
We propose a first adaptation of image-to-image translation networks, that have successfully been used in the 2D domain, to 3D face geometry.
arXiv Detail & Related papers (2021-12-01T14:42:03Z) - TANet: A new Paradigm for Global Face Super-resolution via
Transformer-CNN Aggregation Network [72.41798177302175]
We propose a novel paradigm based on the self-attention mechanism (i.e., the core of Transformer) to fully explore the representation capacity of the facial structure feature.
Specifically, we design a Transformer-CNN aggregation network (TANet) consisting of two paths, in which one path uses CNNs responsible for restoring fine-grained facial details.
By aggregating the features from the above two paths, the consistency of global facial structure and fidelity of local facial detail restoration are strengthened simultaneously.
arXiv Detail & Related papers (2021-09-16T18:15:07Z) - HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping [116.1022638063613]
We propose HifiFace, which can preserve the face shape of the source face and generate photo-realistic results.
We introduce the Semantic Facial Fusion module to optimize the combination of encoder and decoder features.
arXiv Detail & Related papers (2021-06-18T07:39:09Z) - Facial Attribute Transformers for Precise and Robust Makeup Transfer [79.41060385695977]
We propose a novel Facial Attribute Transformer (FAT) and its variant Spatial FAT for high-quality makeup transfer.
FAT is able to model the semantic correspondences and interactions between the source face and reference face, and then precisely estimate and transfer the facial attributes.
We also integrate thin plate splines (TPS) into FAT, thus creating Spatial FAT, which is the first method that can transfer geometric attributes in addition to color and texture.
arXiv Detail & Related papers (2021-04-07T03:39:02Z) - DotFAN: A Domain-transferred Face Augmentation Network for Pose and
Illumination Invariant Face Recognition [94.96686189033869]
We propose a 3D model-assisted domain-transferred face augmentation network (DotFAN)
DotFAN can generate a series of variants of an input face based on the knowledge distilled from existing rich face datasets collected from other domains.
Experiments show that DotFAN is beneficial for augmenting small face datasets to improve their within-class diversity.
arXiv Detail & Related papers (2020-02-23T08:16:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.