ReliableSwap: Boosting General Face Swapping Via Reliable Supervision
- URL: http://arxiv.org/abs/2306.05356v1
- Date: Thu, 8 Jun 2023 17:01:14 GMT
- Title: ReliableSwap: Boosting General Face Swapping Via Reliable Supervision
- Authors: Ge Yuan, Maomao Li, Yong Zhang, Huicheng Zheng
- Abstract summary: This paper proposes to construct reliable supervision, dubbed cycle triplets, which serves as the image-level guidance when the source identity differs from the target one during training.
Specifically, we use face reenactment and blending techniques to synthesize the swapped face from real images in advance.
Our face swapping framework, named ReliableSwap, can boost the performance of any existing face swapping network with negligible overhead.
- Score: 9.725105108879717
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Almost all advanced face swapping approaches use reconstruction as the proxy
task, i.e., supervision only exists when the target and source belong to the
same person. Otherwise, lacking pixel-level supervision, these methods struggle
for source identity preservation. This paper proposes to construct reliable
supervision, dubbed cycle triplets, which serves as the image-level guidance
when the source identity differs from the target one during training.
Specifically, we use face reenactment and blending techniques to synthesize the
swapped face from real images in advance, where the synthetic face preserves
source identity and target attributes. However, there may be some artifacts in
such a synthetic face. To avoid the potential artifacts and drive the
distribution of the network output close to the natural one, we reversely take
synthetic images as input while the real face as reliable supervision during
the training stage of face swapping. Besides, we empirically find that the
existing methods tend to lose lower-face details like face shape and mouth from
the source. This paper additionally designs a FixerNet, providing
discriminative embeddings of lower faces as an enhancement. Our face swapping
framework, named ReliableSwap, can boost the performance of any existing face
swapping network with negligible overhead. Extensive experiments demonstrate
the efficacy of our ReliableSwap, especially in identity preservation. The
project page is https://reliable-swap.github.io/.
Related papers
- High-Fidelity Face Swapping with Style Blending [16.024260677867076]
We propose an innovative end-to-end framework for high-fidelity face swapping.
First, we introduce a StyleGAN-based facial attributes encoder that extracts essential features from faces and inverts them into a latent style code.
Second, we introduce an attention-based style blending module to effectively transfer Face IDs from source to target.
arXiv Detail & Related papers (2023-12-17T23:22:37Z) - BlendFace: Re-designing Identity Encoders for Face-Swapping [2.320417845168326]
BlendFace is a novel identity encoder for face-swapping.
It disentangles identity features into generators and guides generators properly as an identity loss function.
Extensive experiments demonstrate that BlendFace improves the identity-attribute disentanglement in face-swapping models.
arXiv Detail & Related papers (2023-07-20T13:17:30Z) - Face Transformer: Towards High Fidelity and Accurate Face Swapping [54.737909435708936]
Face swapping aims to generate swapped images that fuse the identity of source faces and the attributes of target faces.
This paper presents Face Transformer, a novel face swapping network that can accurately preserve source identities and target attributes simultaneously.
arXiv Detail & Related papers (2023-04-05T15:51:44Z) - FlowFace: Semantic Flow-guided Shape-aware Face Swapping [43.166181219154936]
We propose a semantic flow-guided two-stage framework for shape-aware face swapping, namely FlowFace.
Our FlowFace consists of a face reshaping network and a face swapping network.
We employ a pre-trained face masked autoencoder to extract facial features from both the source face and the target face.
arXiv Detail & Related papers (2022-12-06T07:23:39Z) - UnGANable: Defending Against GAN-based Face Manipulation [69.90981797810348]
Deepfakes pose severe threats of visual misinformation to our society.
One representative deepfake application is face manipulation that modifies a victim's facial attributes in an image.
We propose the first defense system, namely UnGANable, against GAN-inversion-based face manipulation.
arXiv Detail & Related papers (2022-10-03T14:20:01Z) - Restricted Black-box Adversarial Attack Against DeepFake Face Swapping [70.82017781235535]
We introduce a practical adversarial attack that does not require any queries to the facial image forgery model.
Our method is built on a substitute model persuing for face reconstruction and then transfers adversarial examples from the substitute model directly to inaccessible black-box DeepFake models.
arXiv Detail & Related papers (2022-04-26T14:36:06Z) - HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping [116.1022638063613]
We propose HifiFace, which can preserve the face shape of the source face and generate photo-realistic results.
We introduce the Semantic Facial Fusion module to optimize the combination of encoder and decoder features.
arXiv Detail & Related papers (2021-06-18T07:39:09Z) - SimSwap: An Efficient Framework For High Fidelity Face Swapping [43.59969679039686]
We propose an efficient framework, called Simple Swap (SimSwap), aiming for generalized and high fidelity face swapping.
Our framework is capable of transferring the identity of an arbitrary source face into an arbitrary target face while preserving the attributes of the target face.
Experiments on wild faces demonstrate that our SimSwap is able to achieve competitive identity performance while preserving attributes better than previous state-of-the-art methods.
arXiv Detail & Related papers (2021-06-11T12:23:10Z) - One Shot Face Swapping on Megapixels [65.47443090320955]
This paper proposes the first Megapixel level method for one shot Face Swapping (or MegaFS for short)
Complete face representation, stable training, and limited memory usage are the three novel contributions to the success of our method.
arXiv Detail & Related papers (2021-05-11T10:41:47Z) - FaceController: Controllable Attribute Editing for Face in the Wild [74.56117807309576]
We propose a simple feed-forward network to generate high-fidelity manipulated faces.
By simply employing some existing and easy-obtainable prior information, our method can control, transfer, and edit diverse attributes of faces in the wild.
In our method, we decouple identity, expression, pose, and illumination using 3D priors; separate texture and colors by using region-wise style codes.
arXiv Detail & Related papers (2021-02-23T02:47:28Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.