DeepfakeBench-MM: A Comprehensive Benchmark for Multimodal Deepfake Detection

• URL: http://arxiv.org/abs/2510.22622v1
• Date: Sun, 26 Oct 2025 10:40:52 GMT
• Title: DeepfakeBench-MM: A Comprehensive Benchmark for Multimodal Deepfake Detection
• Authors: Kangran Zhao, Yupeng Chen, Xiaoyu Zhang, Yize Chen, Weinan Guan, Baicheng Chen, Chengzhe Sun, Soumyya Kanti Datta, Qingshan Liu, Siwei Lyu, Baoyuan Wu,
• Abstract summary: misuse of advanced generative AI models has resulted in the proliferation of falsified data.<n>Mega-MMDF is a large-scale, diverse, and high-quality dataset for multimodal deepfake detection.<n>DeepfakeBench-MM is the first unified benchmark for multimodal deepfake detection.
• Score: 52.13851094326683
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The misuse of advanced generative AI models has resulted in the widespread proliferation of falsified data, particularly forged human-centric audiovisual content, which poses substantial societal risks (e.g., financial fraud and social instability). In response to this growing threat, several works have preliminarily explored countermeasures. However, the lack of sufficient and diverse training data, along with the absence of a standardized benchmark, hinder deeper exploration. To address this challenge, we first build Mega-MMDF, a large-scale, diverse, and high-quality dataset for multimodal deepfake detection. Specifically, we employ 21 forgery pipelines through the combination of 10 audio forgery methods, 12 visual forgery methods, and 6 audio-driven face reenactment methods. Mega-MMDF currently contains 0.1 million real samples and 1.1 million forged samples, making it one of the largest and most diverse multimodal deepfake datasets, with plans for continuous expansion. Building on it, we present DeepfakeBench-MM, the first unified benchmark for multimodal deepfake detection. It establishes standardized protocols across the entire detection pipeline and serves as a versatile platform for evaluating existing methods as well as exploring novel approaches. DeepfakeBench-MM currently supports 5 datasets and 11 multimodal deepfake detectors. Furthermore, our comprehensive evaluations and in-depth analyses uncover several key findings from multiple perspectives (e.g., augmentation, stacked forgery). We believe that DeepfakeBench-MM, together with our large-scale Mega-MMDF, will serve as foundational infrastructures for advancing multimodal deepfake detection.

Related papers

• DDL: A Large-Scale Datasets for Deepfake Detection and Localization in Diversified Real-World Scenarios [51.916287988122406]
  We present a novel large-scale deepfake detection and localization (textbfDDL) dataset containing over $textbf1.4M+$ forged samples.<n>Our DDL not only provides a more challenging benchmark for complex real-world forgeries but also offers crucial support for building next-generation deepfake detection, localization, and interpretability methods.
  arXiv  Detail & Related papers  (2025-06-29T15:29:03Z)
• DFBench: Benchmarking Deepfake Image Detection Capability of Large Multimodal Models [43.86847047796023]
  Current deepfake detection methods often depend on datasets with limited generation models and content diversity.<n>We present textbfDFBench, a large-scale DeepFake Benchmark featuring 540,000 images across real, AI-edited, and AI-generated content.<n>We propose textbfMoA-DF, Mixture of Agents for DeepFake detection, leveraging a combined probability strategy from multiple LMMs.
  arXiv  Detail & Related papers  (2025-06-03T15:45:41Z)
• Can Multi-modal (reasoning) LLMs work as deepfake detectors? [6.36797761822772]
  We benchmark 12 latest multi-modal LLMs against traditional deepfake detection methods across multiple datasets.<n>Our findings indicate that best multi-modal LLMs achieve competitive performance with promising generalization ability with zero shot.<n>This study highlights the potential of integrating multi-modal reasoning in future deepfake detection frameworks.
  arXiv  Detail & Related papers  (2025-03-25T21:47:29Z)
• Deepfake Media Generation and Detection in the Generative AI Era: A Survey and Outlook [101.30779332427217]
  We survey deepfake generation and detection techniques, including the most recent developments in the field.<n>We identify various kinds of deepfakes, according to the procedure used to alter or generate the fake content.<n>We develop a novel multimodal benchmark to evaluate deepfake detectors on out-of-distribution content.
  arXiv  Detail & Related papers  (2024-11-29T08:29:25Z)
• 1M-Deepfakes Detection Challenge [31.994908331728958]
  The 1M-Deepfakes Detection Challenge is designed to engage the research community in developing advanced methods for detecting and localizing deepfake manipulations. The participants can access the AV-Deepfake1M dataset and are required to submit their inference results for evaluation. The methodologies developed through the challenge will contribute to the development of next-generation deepfake detection and localization systems.
  arXiv  Detail & Related papers  (2024-09-11T03:43:53Z)
• Contextual Cross-Modal Attention for Audio-Visual Deepfake Detection and Localization [3.9440964696313485]
  In the digital age, the emergence of deepfakes and synthetic media presents a significant threat to societal and political integrity. Deepfakes based on multi-modal manipulation, such as audio-visual, are more realistic and pose a greater threat. We propose a novel multi-modal attention framework based on recurrent neural networks (RNNs) that leverages contextual information for audio-visual deepfake detection.
  arXiv  Detail & Related papers  (2024-08-02T18:45:01Z)
• DF40: Toward Next-Generation Deepfake Detection [62.073997142001424]
  existing works identify top-notch detection algorithms and models by adhering to the common practice: training detectors on one specific dataset and testing them on other prevalent deepfake datasets. But can these stand-out "winners" be truly applied to tackle the myriad of realistic and diverse deepfakes lurking in the real world? We construct a highly diverse deepfake detection dataset called DF40, which comprises 40 distinct deepfake techniques.
  arXiv  Detail & Related papers  (2024-06-19T12:35:02Z)
• DeepfakeBench: A Comprehensive Benchmark of Deepfake Detection [55.70982767084996]
  A critical yet frequently overlooked challenge in the field of deepfake detection is the lack of a standardized, unified, comprehensive benchmark. We present the first comprehensive benchmark for deepfake detection, called DeepfakeBench, which offers three key contributions. DeepfakeBench contains 15 state-of-the-art detection methods, 9CL datasets, a series of deepfake detection evaluation protocols and analysis tools, as well as comprehensive evaluations.
  arXiv  Detail & Related papers  (2023-07-04T01:34:41Z)
• A Continual Deepfake Detection Benchmark: Dataset, Methods, and Essentials [97.69553832500547]
  This paper suggests a continual deepfake detection benchmark (CDDB) over a new collection of deepfakes from both known and unknown generative models. We exploit multiple approaches to adapt multiclass incremental learning methods, commonly used in the continual visual recognition, to the continual deepfake detection problem.
  arXiv  Detail & Related papers  (2022-05-11T13:07:19Z)
• M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection [74.19291916812921]
  forged images generated by Deepfake techniques pose a serious threat to the trustworthiness of digital information. In this paper, we aim to capture the subtle manipulation artifacts at different scales for Deepfake detection. We introduce a high-quality Deepfake dataset, SR-DF, which consists of 4,000 DeepFake videos generated by state-of-the-art face swapping and facial reenactment methods.
  arXiv  Detail & Related papers  (2021-04-20T05:43:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.