Deepfake Media Generation and Detection in the Generative AI Era: A Survey and Outlook

• URL: http://arxiv.org/abs/2411.19537v1
• Date: Fri, 29 Nov 2024 08:29:25 GMT
• Title: Deepfake Media Generation and Detection in the Generative AI Era: A Survey and Outlook
• Authors: Florinel-Alin Croitoru, Andrei-Iulian Hiji, Vlad Hondru, Nicolae Catalin Ristea, Paul Irofti, Marius Popescu, Cristian Rusu, Radu Tudor Ionescu, Fahad Shahbaz Khan, Mubarak Shah,
• Abstract summary: We survey deepfake generation and detection techniques, including the most recent developments in the field. We identify various kinds of deepfakes, according to the procedure used to alter or generate the fake content. We develop a novel multimodal benchmark to evaluate deepfake detectors on out-of-distribution content.
• Score: 101.30779332427217
• License:
• Abstract: With the recent advancements in generative modeling, the realism of deepfake content has been increasing at a steady pace, even reaching the point where people often fail to detect manipulated media content online, thus being deceived into various kinds of scams. In this paper, we survey deepfake generation and detection techniques, including the most recent developments in the field, such as diffusion models and Neural Radiance Fields. Our literature review covers all deepfake media types, comprising image, video, audio and multimodal (audio-visual) content. We identify various kinds of deepfakes, according to the procedure used to alter or generate the fake content. We further construct a taxonomy of deepfake generation and detection methods, illustrating the important groups of methods and the domains where these methods are applied. Next, we gather datasets used for deepfake detection and provide updated rankings of the best performing deepfake detectors on the most popular datasets. In addition, we develop a novel multimodal benchmark to evaluate deepfake detectors on out-of-distribution content. The results indicate that state-of-the-art detectors fail to generalize to deepfake content generated by unseen deepfake generators. Finally, we propose future directions to obtain robust and powerful deepfake detectors. Our project page and new benchmark are available at https://github.com/CroitoruAlin/biodeep.

Related papers

• Contextual Cross-Modal Attention for Audio-Visual Deepfake Detection and Localization [3.9440964696313485]
  In the digital age, the emergence of deepfakes and synthetic media presents a significant threat to societal and political integrity. Deepfakes based on multi-modal manipulation, such as audio-visual, are more realistic and pose a greater threat. We propose a novel multi-modal attention framework based on recurrent neural networks (RNNs) that leverages contextual information for audio-visual deepfake detection.
  arXiv  Detail & Related papers  (2024-08-02T18:45:01Z)
• The Tug-of-War Between Deepfake Generation and Detection [4.62070292702111]
  Multimodal generative models are rapidly evolving, leading to a surge in the generation of realistic video and audio. Deepfake videos, which can convincingly impersonate individuals, have particularly garnered attention due to their potential misuse. This survey paper examines the dual landscape of deepfake video generation and detection, emphasizing the need for effective countermeasures.
  arXiv  Detail & Related papers  (2024-07-08T17:49:41Z)
• DF40: Toward Next-Generation Deepfake Detection [62.073997142001424]
  existing works identify top-notch detection algorithms and models by adhering to the common practice: training detectors on one specific dataset and testing them on other prevalent deepfake datasets. But can these stand-out "winners" be truly applied to tackle the myriad of realistic and diverse deepfakes lurking in the real world? We construct a highly diverse deepfake detection dataset called DF40, which comprises 40 distinct deepfake techniques.
  arXiv  Detail & Related papers  (2024-06-19T12:35:02Z)
• Evolving from Single-modal to Multi-modal Facial Deepfake Detection: A Survey [40.11614155244292]
  As AI-generated media become more realistic, the risk of misuse to spread misinformation and commit identity fraud increases. This work traces the evolution from traditional single-modality methods to sophisticated multi-modal approaches that handle audio-visual and text-visual scenarios. To our knowledge, this is the first survey of its kind.
  arXiv  Detail & Related papers  (2024-06-11T05:48:04Z)
• Deepfake Generation and Detection: A Benchmark and Survey [134.19054491600832]
  Deepfake is a technology dedicated to creating highly realistic facial images and videos under specific conditions. This survey comprehensively reviews the latest developments in deepfake generation and detection. We focus on researching four representative deepfake fields: face swapping, face reenactment, talking face generation, and facial attribute editing.
  arXiv  Detail & Related papers  (2024-03-26T17:12:34Z)
• Leveraging Deep Learning Approaches for Deepfake Detection: A Review [0.0]
  Deepfakes are fabricated media generated by AI that are difficult to set apart from the real media. This paper aims to explore different methodologies with an intention to achieve a cost-effective model.
  arXiv  Detail & Related papers  (2023-04-04T16:04:42Z)
• A Continual Deepfake Detection Benchmark: Dataset, Methods, and Essentials [97.69553832500547]
  This paper suggests a continual deepfake detection benchmark (CDDB) over a new collection of deepfakes from both known and unknown generative models. We exploit multiple approaches to adapt multiclass incremental learning methods, commonly used in the continual visual recognition, to the continual deepfake detection problem.
  arXiv  Detail & Related papers  (2022-05-11T13:07:19Z)
• Voice-Face Homogeneity Tells Deepfake [56.334968246631725]
  Existing detection approaches contribute to exploring the specific artifacts in deepfake videos. We propose to perform the deepfake detection from an unexplored voice-face matching view. Our model obtains significantly improved performance as compared to other state-of-the-art competitors.
  arXiv  Detail & Related papers  (2022-03-04T09:08:50Z)
• Deepfakes Generation and Detection: State-of-the-art, open challenges, countermeasures, and way forward [2.15242029196761]
  It is possible to generate deepfakes to disseminate disinformation, revenge porn, financial frauds, hoaxes, and to disrupt government functioning. No attempt has been made to review approaches for detection and generation of both audio and video deepfakes. This paper provides a comprehensive review and detailed analysis of existing tools and machine learning (ML) based approaches for deepfake generation.
  arXiv  Detail & Related papers  (2021-02-25T18:26:50Z)
• WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection [82.42495493102805]
  We introduce a new dataset WildDeepfake which consists of 7,314 face sequences extracted from 707 deepfake videos collected completely from the internet. We conduct a systematic evaluation of a set of baseline detection networks on both existing and our WildDeepfake datasets, and show that WildDeepfake is indeed a more challenging dataset, where the detection performance can decrease drastically.
  arXiv  Detail & Related papers  (2021-01-05T11:10:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.