AOT: Appearance Optimal Transport Based Identity Swapping for Forgery Detection

• URL: http://arxiv.org/abs/2011.02674v1
• Date: Thu, 5 Nov 2020 06:17:04 GMT
• Title: AOT: Appearance Optimal Transport Based Identity Swapping for Forgery Detection
• Authors: Hao Zhu, Chaoyou Fu, Qianyi Wu, Wayne Wu, Chen Qian, Ran He
• Abstract summary: We provide a new identity swapping algorithm with large differences in appearance for face forgery detection. The appearance gaps mainly arise from the large discrepancies in illuminations and skin colors. A discriminator is introduced to distinguish the fake parts from a mix of real and fake image patches.
• Score: 76.7063732501752
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent studies have shown that the performance of forgery detection can be improved with diverse and challenging Deepfakes datasets. However, due to the lack of Deepfakes datasets with large variance in appearance, which can be hardly produced by recent identity swapping methods, the detection algorithm may fail in this situation. In this work, we provide a new identity swapping algorithm with large differences in appearance for face forgery detection. The appearance gaps mainly arise from the large discrepancies in illuminations and skin colors that widely exist in real-world scenarios. However, due to the difficulties of modeling the complex appearance mapping, it is challenging to transfer fine-grained appearances adaptively while preserving identity traits. This paper formulates appearance mapping as an optimal transport problem and proposes an Appearance Optimal Transport model (AOT) to formulate it in both latent and pixel space. Specifically, a relighting generator is designed to simulate the optimal transport plan. It is solved via minimizing the Wasserstein distance of the learned features in the latent space, enabling better performance and less computation than conventional optimization. To further refine the solution of the optimal transport plan, we develop a segmentation game to minimize the Wasserstein distance in the pixel space. A discriminator is introduced to distinguish the fake parts from a mix of real and fake image patches. Extensive experiments reveal that the superiority of our method when compared with state-of-the-art methods and the ability of our generated data to improve the performance of face forgery detection.

Related papers

• Geometry-aware Feature Matching for Large-Scale Structure from Motion [10.645087195983201]
  We introduce geometry cues in addition to color cues to fill gaps when there is less overlap in large-scale scenarios. Our approach ensures that the denser correspondences from detector-free methods are geometrically consistent and more accurate. It outperforms state-of-the-art feature matching methods on benchmark datasets.
  arXiv  Detail & Related papers  (2024-09-03T21:41:35Z)
• Multi-Feature Aggregation in Diffusion Models for Enhanced Face Super-Resolution [6.055006354743854]
  We develop an algorithm that utilize a low-resolution image combined with features extracted from multiple low-quality images to generate a super-resolved image. Unlike other algorithms, our approach recovers facial features without explicitly providing attribute information. This is the first time multi-features combined with low-resolution images are used as conditioners to generate more reliable super-resolution images.
  arXiv  Detail & Related papers  (2024-08-27T20:08:33Z)
• GenFace: A Large-Scale Fine-Grained Face Forgery Benchmark and Cross Appearance-Edge Learning [50.7702397913573]
  The rapid advancement of photorealistic generators has reached a critical juncture where the discrepancy between authentic and manipulated images is increasingly indistinguishable. Although there have been a number of publicly available face forgery datasets, the forgery faces are mostly generated using GAN-based synthesis technology. We propose a large-scale, diverse, and fine-grained high-fidelity dataset, namely GenFace, to facilitate the advancement of deepfake detection.
  arXiv  Detail & Related papers  (2024-02-03T03:13:50Z)
• DeepFidelity: Perceptual Forgery Fidelity Assessment for Deepfake Detection [67.3143177137102]
  Deepfake detection refers to detecting artificially generated or edited faces in images or videos. We propose a novel Deepfake detection framework named DeepFidelity to adaptively distinguish real and fake faces.
  arXiv  Detail & Related papers  (2023-12-07T07:19:45Z)
• Diff-DOPE: Differentiable Deep Object Pose Estimation [29.703385848843414]
  We introduce Diff-DOPE, a 6-DoF pose refiner that takes as input an image, a 3D textured model of an object, and an initial pose of the object. The method uses differentiable rendering to update the object pose to minimize the visual error between the image and the projection of the model. We show that this simple, yet effective, idea is able to achieve state-of-the-art results on pose estimation datasets.
  arXiv  Detail & Related papers  (2023-09-30T18:52:57Z)
• COMICS: End-to-end Bi-grained Contrastive Learning for Multi-face Forgery Detection [56.7599217711363]
  Face forgery recognition methods can only process one face at a time. Most face forgery recognition methods can only process one face at a time. We propose COMICS, an end-to-end framework for multi-face forgery detection.
  arXiv  Detail & Related papers  (2023-08-03T03:37:13Z)
• Unpaired Image Super-Resolution with Optimal Transport Maps [128.1189695209663]
  Real-world image super-resolution (SR) tasks often do not have paired datasets limiting the application of supervised techniques. We propose an algorithm for unpaired SR which learns an unbiased OT map for the perceptual transport cost. Our algorithm provides nearly state-of-the-art performance on the large-scale unpaired AIM-19 dataset.
  arXiv  Detail & Related papers  (2022-02-02T16:21:20Z)
• MogFace: Rethinking Scale Augmentation on the Face Detector [17.570686622370403]
  We investigate the difference among the previous solutions, including the fore-ground and back-ground information of an image and the scale information. We propose a Selective Scale Enhancement (SSE) strategy which can assimilate these two information efficiently and simultaneously. Our method achieves state-of-the-art detection performance on all common face detection benchmarks.
  arXiv  Detail & Related papers  (2021-03-20T09:17:04Z)
• Adversarial Semantic Data Augmentation for Human Pose Estimation [96.75411357541438]
  We propose Semantic Data Augmentation (SDA), a method that augments images by pasting segmented body parts with various semantic granularity. We also propose Adversarial Semantic Data Augmentation (ASDA), which exploits a generative network to dynamiclly predict tailored pasting configuration. State-of-the-art results are achieved on challenging benchmarks.
  arXiv  Detail & Related papers  (2020-08-03T07:56:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.