GazeForensics: DeepFake Detection via Gaze-guided Spatial Inconsistency Learning

• URL: http://arxiv.org/abs/2311.07075v2
• Date: Wed, 22 Nov 2023 23:49:58 GMT
• Title: GazeForensics: DeepFake Detection via Gaze-guided Spatial Inconsistency Learning
• Authors: Qinlin He, Chunlei Peng, Decheng Liu, Nannan Wang, Xinbo Gao
• Abstract summary: We introduce GazeForensics, an innovative DeepFake detection method that utilizes gaze representation obtained from a 3D gaze estimation model. Experiment results reveal that our proposed GazeForensics outperforms the current state-of-the-art methods.
• Score: 63.547321642941974
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: DeepFake detection is pivotal in personal privacy and public safety. With the iterative advancement of DeepFake techniques, high-quality forged videos and images are becoming increasingly deceptive. Prior research has seen numerous attempts by scholars to incorporate biometric features into the field of DeepFake detection. However, traditional biometric-based approaches tend to segregate biometric features from general ones and freeze the biometric feature extractor. These approaches resulted in the exclusion of valuable general features, potentially leading to a performance decline and, consequently, a failure to fully exploit the potential of biometric information in assisting DeepFake detection. Moreover, insufficient attention has been dedicated to scrutinizing gaze authenticity within the realm of DeepFake detection in recent years. In this paper, we introduce GazeForensics, an innovative DeepFake detection method that utilizes gaze representation obtained from a 3D gaze estimation model to regularize the corresponding representation within our DeepFake detection model, while concurrently integrating general features to further enhance the performance of our model. Experiment results reveal that our proposed GazeForensics outperforms the current state-of-the-art methods.

Related papers

• DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusion [94.46904504076124]
  Deepfake technology has made face swapping highly realistic, raising concerns about the malicious use of fabricated facial content. Existing methods often struggle to generalize to unseen domains due to the diverse nature of facial manipulations. We introduce DiffusionFake, a novel framework that reverses the generative process of face forgeries to enhance the generalization of detection models.
  arXiv  Detail & Related papers  (2024-10-06T06:22:43Z)
• Semantics-Oriented Multitask Learning for DeepFake Detection: A Joint Embedding Approach [77.65459419417533]
  We propose an automatic dataset expansion technique to support semantics-oriented DeepFake detection tasks. We also resort to joint embedding of face images and their corresponding labels for prediction. Our method improves the generalizability of DeepFake detection and renders some degree of model interpretation by providing human-understandable explanations.
  arXiv  Detail & Related papers  (2024-08-29T07:11:50Z)
• Masked Conditional Diffusion Model for Enhancing Deepfake Detection [20.018495944984355]
  We propose a Masked Conditional Diffusion Model (MCDM) for enhancing deepfake detection. It generates a variety of forged faces from a masked pristine one, encouraging the deepfake detection model to learn generic and robust representations.
  arXiv  Detail & Related papers  (2024-02-01T12:06:55Z)
• DeepFidelity: Perceptual Forgery Fidelity Assessment for Deepfake Detection [67.3143177137102]
  Deepfake detection refers to detecting artificially generated or edited faces in images or videos. We propose a novel Deepfake detection framework named DeepFidelity to adaptively distinguish real and fake faces.
  arXiv  Detail & Related papers  (2023-12-07T07:19:45Z)
• Facial Forgery-based Deepfake Detection using Fine-Grained Features [7.378937711027777]
  Facial forgery by deepfakes has caused major security risks and raised severe societal concerns. We formulate deepfake detection as a fine-grained classification problem and propose a new fine-grained solution to it. Our method is based on learning subtle and generalizable features by effectively suppressing background noise and learning discriminative features at various scales for deepfake detection.
  arXiv  Detail & Related papers  (2023-10-10T21:30:05Z)
• Attention Consistency Refined Masked Frequency Forgery Representation for Generalizing Face Forgery Detection [96.539862328788]
  Existing forgery detection methods suffer from unsatisfactory generalization ability to determine the authenticity in the unseen domain. We propose a novel Attention Consistency Refined masked frequency forgery representation model toward generalizing face forgery detection algorithm (ACMF) Experiment results on several public face forgery datasets demonstrate the superior performance of the proposed method compared with the state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-07-21T08:58:49Z)
• SeeABLE: Soft Discrepancies and Bounded Contrastive Learning for Exposing Deepfakes [7.553507857251396]
  We propose a novel deepfake detector, called SeeABLE, that formalizes the detection problem as a (one-class) out-of-distribution detection task. SeeABLE pushes perturbed faces towards predefined prototypes using a novel regression-based bounded contrastive loss. We show that our model convincingly outperforms competing state-of-the-art detectors, while exhibiting highly encouraging generalization capabilities.
  arXiv  Detail & Related papers  (2022-11-21T09:38:30Z)
• Deepfake Detection for Facial Images with Facemasks [17.238556058316412]
  We thoroughly evaluate the performance of state-of-the-art deepfake detection models on the deepfakes withthe facemask. We propose two approaches to enhance themasked deepfakes detection:face-patchandface-crop.
  arXiv  Detail & Related papers  (2022-02-23T09:01:27Z)
• DeepFake Detection with Inconsistent Head Poses: Reproducibility and Analysis [0.0]
  We analyze an existing DeepFake detection technique based on head pose estimation. Our results correct the current literature's perception of state of the art performance for DeepFake detection.
  arXiv  Detail & Related papers  (2021-08-28T22:56:09Z)
• Towards Generalizable and Robust Face Manipulation Detection via Bag-of-local-feature [55.47546606878931]
  We propose a novel method for face manipulation detection, which can improve the generalization ability and robustness by bag-of-local-feature. Specifically, we extend Transformers using bag-of-feature approach to encode inter-patch relationships, allowing it to learn local forgery features without any explicit supervision.
  arXiv  Detail & Related papers  (2021-03-14T12:50:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.