Where Do Deep Fakes Look? Synthetic Face Detection via Gaze Tracking

• URL: http://arxiv.org/abs/2101.01165v1
• Date: Mon, 4 Jan 2021 18:54:46 GMT
• Title: Where Do Deep Fakes Look? Synthetic Face Detection via Gaze Tracking
• Authors: Ilke Demir and Umur A. Ciftci
• Abstract summary: We propose several prominent eye and gaze features that deep fakes exhibit differently. Second, we compile those features into signatures and analyze and compare those of real and fake videos. Third, we generalize this formulation to deep fake detection problem by a deep neural network.
• Score: 8.473714899301601
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Following the recent initiatives for the democratization of AI, deep fake generators have become increasingly popular and accessible, causing dystopian scenarios towards social erosion of trust. A particular domain, such as biological signals, attracted attention towards detection methods that are capable of exploiting authenticity signatures in real videos that are not yet faked by generative approaches. In this paper, we first propose several prominent eye and gaze features that deep fakes exhibit differently. Second, we compile those features into signatures and analyze and compare those of real and fake videos, formulating geometric, visual, metric, temporal, and spectral variations. Third, we generalize this formulation to deep fake detection problem by a deep neural network, to classify any video in the wild as fake or real. We evaluate our approach on several deep fake datasets, achieving 89.79\% accuracy on FaceForensics++, 80.0\% on Deep Fakes (in the wild), and 88.35\% on CelebDF datasets. We conduct ablation studies involving different features, architectures, sequence durations, and post-processing artifacts. Our analysis concludes with 6.29\% improved accuracy over complex network architectures without the proposed gaze signatures.

Related papers

• Deepfake detection in videos with multiple faces using geometric-fakeness features [79.16635054977068]
  Deepfakes of victims or public figures can be used by fraudsters for blackmailing, extorsion and financial fraud. In our research we propose to use geometric-fakeness features (GFF) that characterize a dynamic degree of a face presence in a video. We employ our approach to analyze videos with multiple faces that are simultaneously present in a video.
  arXiv  Detail & Related papers  (2024-10-10T13:10:34Z)
• Shaking the Fake: Detecting Deepfake Videos in Real Time via Active Probes [3.6308756891251392]
  Real-time deepfake, a type of generative AI, is capable of "creating" non-existing contents (e.g., swapping one's face with another) in a video. It has been misused to produce deepfake videos for malicious purposes, including financial scams and political misinformation. We propose SFake, a new real-time deepfake detection method that exploits deepfake models' inability to adapt to physical interference.
  arXiv  Detail & Related papers  (2024-09-17T04:58:30Z)
• DeepFidelity: Perceptual Forgery Fidelity Assessment for Deepfake Detection [67.3143177137102]
  Deepfake detection refers to detecting artificially generated or edited faces in images or videos. We propose a novel Deepfake detection framework named DeepFidelity to adaptively distinguish real and fake faces.
  arXiv  Detail & Related papers  (2023-12-07T07:19:45Z)
• How Do Deepfakes Move? Motion Magnification for Deepfake Source Detection [4.567475511774088]
  We build a generalized deepfake source detector based on sub-muscular motion in faces. Our approach exploits the difference between real motion and the amplified GAN fingerprints. We evaluate our approach on two multi-source datasets.
  arXiv  Detail & Related papers  (2022-12-28T18:59:21Z)
• DeePhy: On Deepfake Phylogeny [58.01631614114075]
  DeePhy is a novel Deepfake Phylogeny dataset which consists of 5040 deepfake videos generated using three different generation techniques. We present the benchmark on DeePhy dataset using six deepfake detection algorithms.
  arXiv  Detail & Related papers  (2022-09-19T15:30:33Z)
• Voice-Face Homogeneity Tells Deepfake [56.334968246631725]
  Existing detection approaches contribute to exploring the specific artifacts in deepfake videos. We propose to perform the deepfake detection from an unexplored voice-face matching view. Our model obtains significantly improved performance as compared to other state-of-the-art competitors.
  arXiv  Detail & Related papers  (2022-03-04T09:08:50Z)
• Beyond the Spectrum: Detecting Deepfakes via Re-Synthesis [69.09526348527203]
  Deep generative models have led to highly realistic media, known as deepfakes, that are commonly indistinguishable from real to human eyes. We propose a novel fake detection that is designed to re-synthesize testing images and extract visual cues for detection. We demonstrate the improved effectiveness, cross-GAN generalization, and robustness against perturbations of our approach in a variety of detection scenarios.
  arXiv  Detail & Related papers  (2021-05-29T21:22:24Z)
• Identity-Driven DeepFake Detection [91.0504621868628]
  Identity-Driven DeepFake Detection takes as input the suspect image/video as well as the target identity information. We output a decision on whether the identity in the suspect image/video is the same as the target identity. We present a simple identity-based detection algorithm called the OuterFace, which may serve as a baseline for further research.
  arXiv  Detail & Related papers  (2020-12-07T18:59:08Z)
• How Do the Hearts of Deep Fakes Beat? Deep Fake Source Detection via Interpreting Residuals with Biological Signals [9.918684475252636]
  We propose an approach not only to separate deep fakes from real, but also to discover the specific generative model behind a deep fake. Our results indicate that our approach can detect fake videos with 97.29% accuracy, and the source model with 93.39% accuracy.
  arXiv  Detail & Related papers  (2020-08-26T03:35:47Z)
• DeepFakes Evolution: Analysis of Facial Regions and Fake Detection Performance [3.441021278275805]
  This study provides an exhaustive analysis of both 1st and 2nd DeepFake generations in terms of facial regions and fake detection performance. We highlight the poor fake detection results achieved even by the strongest state-of-the-art fake detectors in the latest DeepFake databases of the 2nd generation.
  arXiv  Detail & Related papers  (2020-04-16T08:49:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.