BlendFace: Re-designing Identity Encoders for Face-Swapping
- URL: http://arxiv.org/abs/2307.10854v1
- Date: Thu, 20 Jul 2023 13:17:30 GMT
- Title: BlendFace: Re-designing Identity Encoders for Face-Swapping
- Authors: Kaede Shiohara, Xingchao Yang, Takafumi Taketomi
- Abstract summary: BlendFace is a novel identity encoder for face-swapping.
It disentangles identity features into generators and guides generators properly as an identity loss function.
Extensive experiments demonstrate that BlendFace improves the identity-attribute disentanglement in face-swapping models.
- Score: 2.320417845168326
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The great advancements of generative adversarial networks and face
recognition models in computer vision have made it possible to swap identities
on images from single sources. Although a lot of studies seems to have proposed
almost satisfactory solutions, we notice previous methods still suffer from an
identity-attribute entanglement that causes undesired attributes swapping
because widely used identity encoders, eg, ArcFace, have some crucial attribute
biases owing to their pretraining on face recognition tasks. To address this
issue, we design BlendFace, a novel identity encoder for face-swapping. The key
idea behind BlendFace is training face recognition models on blended images
whose attributes are replaced with those of another mitigates inter-personal
biases such as hairsyles. BlendFace feeds disentangled identity features into
generators and guides generators properly as an identity loss function.
Extensive experiments demonstrate that BlendFace improves the
identity-attribute disentanglement in face-swapping models, maintaining a
comparable quantitative performance to previous methods.
Related papers
- StableIdentity: Inserting Anybody into Anywhere at First Sight [57.99693188913382]
We propose StableIdentity, which allows identity-consistent recontextualization with just one face image.
We are the first to directly inject the identity learned from a single image into video/3D generation without finetuning.
arXiv Detail & Related papers (2024-01-29T09:06:15Z) - High-Fidelity Face Swapping with Style Blending [16.024260677867076]
We propose an innovative end-to-end framework for high-fidelity face swapping.
First, we introduce a StyleGAN-based facial attributes encoder that extracts essential features from faces and inverts them into a latent style code.
Second, we introduce an attention-based style blending module to effectively transfer Face IDs from source to target.
arXiv Detail & Related papers (2023-12-17T23:22:37Z) - Seeing through the Mask: Multi-task Generative Mask Decoupling Face
Recognition [47.248075664420874]
Current general face recognition system suffers from serious performance degradation when encountering occluded scenes.
This paper proposes a Multi-task gEnerative mask dEcoupling face Recognition (MEER) network to jointly handle these two tasks.
We first present a novel mask decoupling module to disentangle mask and identity information, which makes the network obtain purer identity features from visible facial components.
arXiv Detail & Related papers (2023-11-20T03:23:03Z) - ReliableSwap: Boosting General Face Swapping Via Reliable Supervision [9.725105108879717]
This paper proposes to construct reliable supervision, dubbed cycle triplets, which serves as the image-level guidance when the source identity differs from the target one during training.
Specifically, we use face reenactment and blending techniques to synthesize the swapped face from real images in advance.
Our face swapping framework, named ReliableSwap, can boost the performance of any existing face swapping network with negligible overhead.
arXiv Detail & Related papers (2023-06-08T17:01:14Z) - Face Transformer: Towards High Fidelity and Accurate Face Swapping [54.737909435708936]
Face swapping aims to generate swapped images that fuse the identity of source faces and the attributes of target faces.
This paper presents Face Transformer, a novel face swapping network that can accurately preserve source identities and target attributes simultaneously.
arXiv Detail & Related papers (2023-04-05T15:51:44Z) - FaceMAE: Privacy-Preserving Face Recognition via Masked Autoencoders [81.21440457805932]
We propose a novel framework FaceMAE, where the face privacy and recognition performance are considered simultaneously.
randomly masked face images are used to train the reconstruction module in FaceMAE.
We also perform sufficient privacy-preserving face recognition on several public face datasets.
arXiv Detail & Related papers (2022-05-23T07:19:42Z) - Learning Disentangled Representation for One-shot Progressive Face
Swapping [65.98684203654908]
We present a simple yet efficient method named FaceSwapper, for one-shot face swapping based on Generative Adversarial Networks.
Our method consists of a disentangled representation module and a semantic-guided fusion module.
Our results show that our method achieves state-of-the-art results on benchmark with fewer training samples.
arXiv Detail & Related papers (2022-03-24T11:19:04Z) - A Systematical Solution for Face De-identification [6.244117712209321]
In different tasks, people have various requirements for face de-identification (De-ID)
We propose a systematical solution compatible for these De-ID operations.
Our method can flexibly de-identify the face data in various ways and the processed images have high image quality.
arXiv Detail & Related papers (2021-07-19T02:02:51Z) - HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping [116.1022638063613]
We propose HifiFace, which can preserve the face shape of the source face and generate photo-realistic results.
We introduce the Semantic Facial Fusion module to optimize the combination of encoder and decoder features.
arXiv Detail & Related papers (2021-06-18T07:39:09Z) - DotFAN: A Domain-transferred Face Augmentation Network for Pose and
Illumination Invariant Face Recognition [94.96686189033869]
We propose a 3D model-assisted domain-transferred face augmentation network (DotFAN)
DotFAN can generate a series of variants of an input face based on the knowledge distilled from existing rich face datasets collected from other domains.
Experiments show that DotFAN is beneficial for augmenting small face datasets to improve their within-class diversity.
arXiv Detail & Related papers (2020-02-23T08:16:34Z) - FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping [43.236261887752065]
We propose a novel two-stage framework, called FaceShifter, for high fidelity and occlusion aware face swapping.
In its first stage, our framework generates the swapped face in high-fidelity by exploiting and integrating the target attributes thoroughly and adaptively.
To address the challenging facial synthesiss, we append a second stage consisting of a novel Heuristic Error Acknowledging Refinement Network (HEAR-Net)
arXiv Detail & Related papers (2019-12-31T17:57:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.