GOTCHA: Real-Time Video Deepfake Detection via Challenge-Response

• URL: http://arxiv.org/abs/2210.06186v4
• Date: Thu, 23 May 2024 19:00:04 GMT
• Title: GOTCHA: Real-Time Video Deepfake Detection via Challenge-Response
• Authors: Govind Mittal, Chinmay Hegde, Nasir Memon,
• Abstract summary: We propose a challenge-response approach that establishes authenticity in live settings. We focus on talking-head style video interaction and present a taxonomy of challenges that specifically target inherent limitations of RTDF generation pipelines. The findings underscore the promising potential of challenge-response systems for explainable and scalable real-time deepfake detection.
• Score: 17.117162678626418
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: With the rise of AI-enabled Real-Time Deepfakes (RTDFs), the integrity of online video interactions has become a growing concern. RTDFs have now made it feasible to replace an imposter's face with their victim in live video interactions. Such advancement in deepfakes also coaxes detection to rise to the same standard. However, existing deepfake detection techniques are asynchronous and hence ill-suited for RTDFs. To bridge this gap, we propose a challenge-response approach that establishes authenticity in live settings. We focus on talking-head style video interaction and present a taxonomy of challenges that specifically target inherent limitations of RTDF generation pipelines. We evaluate representative examples from the taxonomy by collecting a unique dataset comprising eight challenges, which consistently and visibly degrades the quality of state-of-the-art deepfake generators. These results are corroborated both by humans and a new automated scoring function, leading to 88.6% and 80.1% AUC, respectively. The findings underscore the promising potential of challenge-response systems for explainable and scalable real-time deepfake detection in practical scenarios. We provide access to data and code at \url{https://github.com/mittalgovind/GOTCHA-Deepfakes}.

Related papers

• Deepfake detection in videos with multiple faces using geometric-fakeness features [79.16635054977068]
  Deepfakes of victims or public figures can be used by fraudsters for blackmailing, extorsion and financial fraud. In our research we propose to use geometric-fakeness features (GFF) that characterize a dynamic degree of a face presence in a video. We employ our approach to analyze videos with multiple faces that are simultaneously present in a video.
  arXiv  Detail & Related papers  (2024-10-10T13:10:34Z)
• Shaking the Fake: Detecting Deepfake Videos in Real Time via Active Probes [3.6308756891251392]
  Real-time deepfake, a type of generative AI, is capable of "creating" non-existing contents (e.g., swapping one's face with another) in a video. It has been misused to produce deepfake videos for malicious purposes, including financial scams and political misinformation. We propose SFake, a new real-time deepfake detection method that exploits deepfake models' inability to adapt to physical interference.
  arXiv  Detail & Related papers  (2024-09-17T04:58:30Z)
• Conditioned Prompt-Optimization for Continual Deepfake Detection [11.634681724245933]
  This paper introduces Prompt2Guard, a novel solution for photorealistic-free continual deepfake detection of images. We leverage a prediction ensembling technique with read-only prompts, mitigating the need for multiple forward passes. Our method exploits a text-prompt conditioning tailored to deepfake detection, which we demonstrate is beneficial in our setting.
  arXiv  Detail & Related papers  (2024-07-31T12:22:57Z)
• DF40: Toward Next-Generation Deepfake Detection [62.073997142001424]
  existing works identify top-notch detection algorithms and models by adhering to the common practice: training detectors on one specific dataset and testing them on other prevalent deepfake datasets. But can these stand-out "winners" be truly applied to tackle the myriad of realistic and diverse deepfakes lurking in the real world? We construct a highly diverse deepfake detection dataset called DF40, which comprises 40 distinct deepfake techniques.
  arXiv  Detail & Related papers  (2024-06-19T12:35:02Z)
• XAI-Based Detection of Adversarial Attacks on Deepfake Detectors [0.0]
  We introduce a novel methodology for identifying adversarial attacks on deepfake detectors using XAI. Our approach contributes not only to the detection of deepfakes but also enhances the understanding of possible adversarial attacks.
  arXiv  Detail & Related papers  (2024-03-05T13:25:30Z)
• NPVForensics: Jointing Non-critical Phonemes and Visemes for Deepfake Detection [50.33525966541906]
  Existing multimodal detection methods capture audio-visual inconsistencies to expose Deepfake videos. We propose a novel Deepfake detection method to mine the correlation between Non-critical Phonemes and Visemes, termed NPVForensics. Our model can be easily adapted to the downstream Deepfake datasets with fine-tuning.
  arXiv  Detail & Related papers  (2023-06-12T06:06:05Z)
• Voice-Face Homogeneity Tells Deepfake [56.334968246631725]
  Existing detection approaches contribute to exploring the specific artifacts in deepfake videos. We propose to perform the deepfake detection from an unexplored voice-face matching view. Our model obtains significantly improved performance as compared to other state-of-the-art competitors.
  arXiv  Detail & Related papers  (2022-03-04T09:08:50Z)
• Beyond the Spectrum: Detecting Deepfakes via Re-Synthesis [69.09526348527203]
  Deep generative models have led to highly realistic media, known as deepfakes, that are commonly indistinguishable from real to human eyes. We propose a novel fake detection that is designed to re-synthesize testing images and extract visual cues for detection. We demonstrate the improved effectiveness, cross-GAN generalization, and robustness against perturbations of our approach in a variety of detection scenarios.
  arXiv  Detail & Related papers  (2021-05-29T21:22:24Z)
• WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection [82.42495493102805]
  We introduce a new dataset WildDeepfake which consists of 7,314 face sequences extracted from 707 deepfake videos collected completely from the internet. We conduct a systematic evaluation of a set of baseline detection networks on both existing and our WildDeepfake datasets, and show that WildDeepfake is indeed a more challenging dataset, where the detection performance can decrease drastically.
  arXiv  Detail & Related papers  (2021-01-05T11:10:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.