ReliableSwap: Boosting General Face Swapping Via Reliable Supervision

• URL: http://arxiv.org/abs/2306.05356v1
• Date: Thu, 8 Jun 2023 17:01:14 GMT
• Title: ReliableSwap: Boosting General Face Swapping Via Reliable Supervision
• Authors: Ge Yuan, Maomao Li, Yong Zhang, Huicheng Zheng
• Abstract summary: This paper proposes to construct reliable supervision, dubbed cycle triplets, which serves as the image-level guidance when the source identity differs from the target one during training. Specifically, we use face reenactment and blending techniques to synthesize the swapped face from real images in advance. Our face swapping framework, named ReliableSwap, can boost the performance of any existing face swapping network with negligible overhead.
• Score: 9.725105108879717
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Almost all advanced face swapping approaches use reconstruction as the proxy task, i.e., supervision only exists when the target and source belong to the same person. Otherwise, lacking pixel-level supervision, these methods struggle for source identity preservation. This paper proposes to construct reliable supervision, dubbed cycle triplets, which serves as the image-level guidance when the source identity differs from the target one during training. Specifically, we use face reenactment and blending techniques to synthesize the swapped face from real images in advance, where the synthetic face preserves source identity and target attributes. However, there may be some artifacts in such a synthetic face. To avoid the potential artifacts and drive the distribution of the network output close to the natural one, we reversely take synthetic images as input while the real face as reliable supervision during the training stage of face swapping. Besides, we empirically find that the existing methods tend to lose lower-face details like face shape and mouth from the source. This paper additionally designs a FixerNet, providing discriminative embeddings of lower faces as an enhancement. Our face swapping framework, named ReliableSwap, can boost the performance of any existing face swapping network with negligible overhead. Extensive experiments demonstrate the efficacy of our ReliableSwap, especially in identity preservation. The project page is https://reliable-swap.github.io/.

Related papers

• OSDFace: One-Step Diffusion Model for Face Restoration [72.5045389847792]
  Diffusion models have demonstrated impressive performance in face restoration. We propose OSDFace, a novel one-step diffusion model for face restoration. Results demonstrate that OSDFace surpasses current state-of-the-art (SOTA) methods in both visual quality and quantitative metrics.
  arXiv  Detail & Related papers  (2024-11-26T07:07:48Z)
• G2Face: High-Fidelity Reversible Face Anonymization via Generative and Geometric Priors [71.69161292330504]
  Reversible face anonymization seeks to replace sensitive identity information in facial images with synthesized alternatives. This paper introduces Gtextsuperscript2Face, which leverages both generative and geometric priors to enhance identity manipulation. Our method outperforms existing state-of-the-art techniques in face anonymization and recovery, while preserving high data utility.
  arXiv  Detail & Related papers  (2024-08-18T12:36:47Z)
• IDRetracor: Towards Visual Forensics Against Malicious Face Swapping [30.804429527783395]
  Face swapping technique based on deepfake methods poses significant social risks to personal identity security. We propose a novel task named face retracing, which considers retracing the original target face from the given fake one via inverse mapping. We show that the IDRetracor exhibits promising retracing performance from both quantitative and qualitative perspectives.
  arXiv  Detail & Related papers  (2024-08-13T04:53:48Z)
• High-Fidelity Face Swapping with Style Blending [16.024260677867076]
  We propose an innovative end-to-end framework for high-fidelity face swapping. First, we introduce a StyleGAN-based facial attributes encoder that extracts essential features from faces and inverts them into a latent style code. Second, we introduce an attention-based style blending module to effectively transfer Face IDs from source to target.
  arXiv  Detail & Related papers  (2023-12-17T23:22:37Z)
• BlendFace: Re-designing Identity Encoders for Face-Swapping [2.320417845168326]
  BlendFace is a novel identity encoder for face-swapping. It disentangles identity features into generators and guides generators properly as an identity loss function. Extensive experiments demonstrate that BlendFace improves the identity-attribute disentanglement in face-swapping models.
  arXiv  Detail & Related papers  (2023-07-20T13:17:30Z)
• Face Transformer: Towards High Fidelity and Accurate Face Swapping [54.737909435708936]
  Face swapping aims to generate swapped images that fuse the identity of source faces and the attributes of target faces. This paper presents Face Transformer, a novel face swapping network that can accurately preserve source identities and target attributes simultaneously.
  arXiv  Detail & Related papers  (2023-04-05T15:51:44Z)
• FlowFace: Semantic Flow-guided Shape-aware Face Swapping [43.166181219154936]
  We propose a semantic flow-guided two-stage framework for shape-aware face swapping, namely FlowFace. Our FlowFace consists of a face reshaping network and a face swapping network. We employ a pre-trained face masked autoencoder to extract facial features from both the source face and the target face.
  arXiv  Detail & Related papers  (2022-12-06T07:23:39Z)
• Restricted Black-box Adversarial Attack Against DeepFake Face Swapping [70.82017781235535]
  We introduce a practical adversarial attack that does not require any queries to the facial image forgery model. Our method is built on a substitute model persuing for face reconstruction and then transfers adversarial examples from the substitute model directly to inaccessible black-box DeepFake models.
  arXiv  Detail & Related papers  (2022-04-26T14:36:06Z)
• HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping [116.1022638063613]
  We propose HifiFace, which can preserve the face shape of the source face and generate photo-realistic results. We introduce the Semantic Facial Fusion module to optimize the combination of encoder and decoder features.
  arXiv  Detail & Related papers  (2021-06-18T07:39:09Z)
• SimSwap: An Efficient Framework For High Fidelity Face Swapping [43.59969679039686]
  We propose an efficient framework, called Simple Swap (SimSwap), aiming for generalized and high fidelity face swapping. Our framework is capable of transferring the identity of an arbitrary source face into an arbitrary target face while preserving the attributes of the target face. Experiments on wild faces demonstrate that our SimSwap is able to achieve competitive identity performance while preserving attributes better than previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-11T12:23:10Z)
• One Shot Face Swapping on Megapixels [65.47443090320955]
  This paper proposes the first Megapixel level method for one shot Face Swapping (or MegaFS for short) Complete face representation, stable training, and limited memory usage are the three novel contributions to the success of our method.
  arXiv  Detail & Related papers  (2021-05-11T10:41:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.