Smooth-Swap: A Simple Enhancement for Face-Swapping with Smoothness

• URL: http://arxiv.org/abs/2112.05907v1
• Date: Sat, 11 Dec 2021 03:26:32 GMT
• Title: Smooth-Swap: A Simple Enhancement for Face-Swapping with Smoothness
• Authors: Jiseob Kim, Jihoon Lee, Byoung-Tak Zhang
• Abstract summary: We propose a new face-swapping model called Smooth-Swap' It focuses on deriving the smoothness of the identity embedding instead of employing complex handcrafted designs. Our model is quantitatively and qualitatively comparable or even superior to existing methods in terms of identity change.
• Score: 18.555874044296463
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, face-swapping models have progressed in generation quality and drawn attention for their applications in privacy protection and entertainment. However, their complex architectures and loss functions often require careful tuning for successful training. In this paper, we propose a new face-swapping model called `Smooth-Swap', which focuses on deriving the smoothness of the identity embedding instead of employing complex handcrafted designs. We postulate that the gist of the difficulty in face-swapping is unstable gradients and it can be resolved by a smooth identity embedder. Smooth-swap adopts an embedder trained using supervised contrastive learning, where we find its improved smoothness allows faster and stable training even with a simple U-Net-based generator and three basic loss functions. Extensive experiments on face-swapping benchmarks (FFHQ, FaceForensics++) and face images in the wild show that our model is also quantitatively and qualitatively comparable or even superior to existing methods in terms of identity change.

Related papers

• OSDFace: One-Step Diffusion Model for Face Restoration [72.5045389847792]
  Diffusion models have demonstrated impressive performance in face restoration. We propose OSDFace, a novel one-step diffusion model for face restoration. Results demonstrate that OSDFace surpasses current state-of-the-art (SOTA) methods in both visual quality and quantitative metrics.
  arXiv  Detail & Related papers  (2024-11-26T07:07:48Z)
• Realistic and Efficient Face Swapping: A Unified Approach with Diffusion Models [69.50286698375386]
  We propose a novel approach that better harnesses diffusion models for face-swapping. We introduce a mask shuffling technique during inpainting training, which allows us to create a so-called universal model for swapping. Ours is a relatively unified approach and so it is resilient to errors in other off-the-shelf models.
  arXiv  Detail & Related papers  (2024-09-11T13:43:53Z)
• High-Fidelity Face Swapping with Style Blending [16.024260677867076]
  We propose an innovative end-to-end framework for high-fidelity face swapping. First, we introduce a StyleGAN-based facial attributes encoder that extracts essential features from faces and inverts them into a latent style code. Second, we introduce an attention-based style blending module to effectively transfer Face IDs from source to target.
  arXiv  Detail & Related papers  (2023-12-17T23:22:37Z)
• Face Transformer: Towards High Fidelity and Accurate Face Swapping [54.737909435708936]
  Face swapping aims to generate swapped images that fuse the identity of source faces and the attributes of target faces. This paper presents Face Transformer, a novel face swapping network that can accurately preserve source identities and target attributes simultaneously.
  arXiv  Detail & Related papers  (2023-04-05T15:51:44Z)
• End-to-end Face-swapping via Adaptive Latent Representation Learning [12.364688530047786]
  This paper proposes a novel and end-to-end integrated framework for high resolution and attribute preservation face swapping. Our framework integrating facial perceiving and blending into the end-to-end training and testing process can achieve high realistic face-swapping on wild faces.
  arXiv  Detail & Related papers  (2023-03-07T19:16:20Z)
• StyleSwap: Style-Based Generator Empowers Robust Face Swapping [90.05775519962303]
  We introduce a concise and effective framework named StyleSwap. Our core idea is to leverage a style-based generator to empower high-fidelity and robust face swapping. We identify that with only minimal modifications, a StyleGAN2 architecture can successfully handle the desired information from both source and target.
  arXiv  Detail & Related papers  (2022-09-27T16:35:16Z)
• HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping [116.1022638063613]
  We propose HifiFace, which can preserve the face shape of the source face and generate photo-realistic results. We introduce the Semantic Facial Fusion module to optimize the combination of encoder and decoder features.
  arXiv  Detail & Related papers  (2021-06-18T07:39:09Z)
• Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection [65.92058628082322]
  Non-parametric face modeling aims to reconstruct 3D face only from images without shape assumptions. This paper presents a novel Learning to Aggregate and Personalize framework for unsupervised robust 3D face modeling.
  arXiv  Detail & Related papers  (2021-06-15T03:10:17Z)
• FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping [43.236261887752065]
  We propose a novel two-stage framework, called FaceShifter, for high fidelity and occlusion aware face swapping. In its first stage, our framework generates the swapped face in high-fidelity by exploiting and integrating the target attributes thoroughly and adaptively. To address the challenging facial synthesiss, we append a second stage consisting of a novel Heuristic Error Acknowledging Refinement Network (HEAR-Net)
  arXiv  Detail & Related papers  (2019-12-31T17:57:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.