Fit for Purpose? Deepfake Detection in the Real World

• URL: http://arxiv.org/abs/2510.16556v2
• Date: Thu, 30 Oct 2025 16:01:55 GMT
• Title: Fit for Purpose? Deepfake Detection in the Real World
• Authors: Guangyu Lin, Li Lin, Christina P. Walker, Daniel S. Schiff, Shu Hu,
• Abstract summary: We introduce the first systematic benchmark based on the Political Deepfakes Incident Database.<n>Our study includes a systematic evaluation of state-of-the-art deepfake detectors across academia, government, and industry.
• Score: 9.009097268891717
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rapid proliferation of AI-generated content, driven by advances in generative adversarial networks, diffusion models, and multimodal large language models, has made the creation and dissemination of synthetic media effortless, heightening the risks of misinformation, particularly political deepfakes that distort truth and undermine trust in political institutions. In turn, governments, research institutions, and industry have strongly promoted deepfake detection initiatives as solutions. Yet, most existing models are trained and validated on synthetic, laboratory-controlled datasets, limiting their generalizability to the kinds of real-world political deepfakes circulating on social platforms that affect the public. In this work, we introduce the first systematic benchmark based on the Political Deepfakes Incident Database, a curated collection of real-world political deepfakes shared on social media since 2018. Our study includes a systematic evaluation of state-of-the-art deepfake detectors across academia, government, and industry. We find that the detectors from academia and government perform relatively poorly. While paid detection tools achieve relatively higher performance than free-access models, all evaluated detectors struggle to generalize effectively to authentic political deepfakes, and are vulnerable to simple manipulations, especially in the video domain. Results urge the need for politically contextualized deepfake detection frameworks to better safeguard the public in real-world settings.

Related papers

• OpenFake: An Open Dataset and Platform Toward Real-World Deepfake Detection [6.949215801100937]
  Deepfakes, synthetic media created using advanced AI techniques, pose a growing threat to information integrity.<n>We present OpenFake, a dataset specifically crafted for benchmarking against modern generative models with high realism.<n> Detectors trained on OpenFake achieve near-perfect in-distribution performance, strong generalization to unseen generators, and high accuracy on a curated in-the-wild social media test set.
  arXiv  Detail & Related papers  (2025-09-11T14:34:22Z)
• SocialDF: Benchmark Dataset and Detection Model for Mitigating Harmful Deepfake Content on Social Media Platforms [0.13194391758295113]
  We introduce SocialDF, a curated dataset reflecting real-world deepfake challenges on social media platforms.<n>This dataset encompasses high-fidelity deepfakes sourced from various online ecosystems.<n>We propose a novel multi-factor detection approach that combines facial recognition, automated speech transcription, and a multi-agent LLM pipeline.
  arXiv  Detail & Related papers  (2025-06-05T19:39:28Z)
• So-Fake: Benchmarking and Explaining Social Media Image Forgery Detection [75.79507634008631]
  We introduce So-Fake-Set, a social media-oriented dataset with over 2 million high-quality images, diverse generative sources, and imagery synthesized using 35 state-of-the-art generative models.<n>We present So-Fake-R1, an advanced vision-language framework that employs reinforcement learning for highly accurate forgery detection, precise localization, and explainable inference through interpretable visual rationales.
  arXiv  Detail & Related papers  (2025-05-24T11:53:35Z)
• Deepfake Media Generation and Detection in the Generative AI Era: A Survey and Outlook [101.30779332427217]
  We survey deepfake generation and detection techniques, including the most recent developments in the field.<n>We identify various kinds of deepfakes, according to the procedure used to alter or generate the fake content.<n>We develop a novel multimodal benchmark to evaluate deepfake detectors on out-of-distribution content.
  arXiv  Detail & Related papers  (2024-11-29T08:29:25Z)
• Understanding Audiovisual Deepfake Detection: Techniques, Challenges, Human Factors and Perceptual Insights [49.81915942821647]
  Deep Learning has been successfully applied in diverse fields, and its impact on deepfake detection is no exception. Deepfakes are fake yet realistic synthetic content that can be used deceitfully for political impersonation, phishing, slandering, or spreading misinformation. This paper aims to improve the effectiveness of deepfake detection strategies and guide future research in cybersecurity and media integrity.
  arXiv  Detail & Related papers  (2024-11-12T09:02:11Z)
• A Survey of Stance Detection on Social Media: New Directions and Perspectives [50.27382951812502]
  stance detection has emerged as a crucial subfield within affective computing. Recent years have seen a surge of research interest in developing effective stance detection methods. This paper provides a comprehensive survey of stance detection techniques on social media.
  arXiv  Detail & Related papers  (2024-09-24T03:06:25Z)
• Merging AI Incidents Research with Political Misinformation Research: Introducing the Political Deepfakes Incidents Database [0.0]
  The project is driven by the rise of generative AI in politics, ongoing policy efforts to address harms, and the need to connect AI incidents and political communication research. The database contains political deepfake content, metadata, and researcher-coded descriptors drawn from political science, public policy, communication, and misinformation studies. It aims to help reveal the prevalence, trends, and impact of political deepfakes, such as those featuring major political figures or events.
  arXiv  Detail & Related papers  (2024-09-05T19:24:38Z)
• Deepfake Media Forensics: State of the Art and Challenges Ahead [51.33414186878676]
  AI-generated synthetic media, also called Deepfakes, have influenced so many domains, from entertainment to cybersecurity. Deepfake detection has become a vital area of research, focusing on identifying subtle inconsistencies and artifacts with machine learning techniques. This paper reviews the primary algorithms that address these challenges, examining their advantages, limitations, and future prospects.
  arXiv  Detail & Related papers  (2024-08-01T08:57:47Z)
• Examining the Implications of Deepfakes for Election Integrity [9.129491613898962]
  It is becoming cheaper to launch disinformation operations at scale using AI-generated content, in particular 'deepfake' technology. We discuss the threats from deepfakes in politics, highlight model specifications underlying different types of deepfake generation methods, and contribute an accessible evaluation of the efficacy of existing detection methods. We highlight the limitations of existing detection mechanisms and discuss the areas where policies and regulations are required to address the challenges of deepfakes.
  arXiv  Detail & Related papers  (2024-06-20T13:15:54Z)
• Deepfake Generation and Detection: A Benchmark and Survey [134.19054491600832]
  Deepfake is a technology dedicated to creating highly realistic facial images and videos under specific conditions. This survey comprehensively reviews the latest developments in deepfake generation and detection. We focus on researching four representative deepfake fields: face swapping, face reenactment, talking face generation, and facial attribute editing.
  arXiv  Detail & Related papers  (2024-03-26T17:12:34Z)
• Leveraging Deep Learning Approaches for Deepfake Detection: A Review [0.0]
  Deepfakes are fabricated media generated by AI that are difficult to set apart from the real media. This paper aims to explore different methodologies with an intention to achieve a cost-effective model.
  arXiv  Detail & Related papers  (2023-04-04T16:04:42Z)
• Leveraging Multi-Source Weak Social Supervision for Early Detection of Fake News [67.53424807783414]
  Social media has greatly enabled people to participate in online activities at an unprecedented rate. This unrestricted access also exacerbates the spread of misinformation and fake news online which might cause confusion and chaos unless being detected early for its mitigation. We jointly leverage the limited amount of clean data along with weak signals from social engagements to train deep neural networks in a meta-learning framework to estimate the quality of different weak instances. Experiments on realworld datasets demonstrate that the proposed framework outperforms state-of-the-art baselines for early detection of fake news without using any user engagements at prediction time.
  arXiv  Detail & Related papers  (2020-04-03T18:26:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.