Investigation of ensemble methods for the detection of deepfake face
manipulations
- URL: http://arxiv.org/abs/2304.07395v1
- Date: Fri, 14 Apr 2023 21:18:51 GMT
- Title: Investigation of ensemble methods for the detection of deepfake face
manipulations
- Authors: Nikolaos Giatsoglou, Symeon Papadopoulos, Ioannis Kompatsiaris
- Abstract summary: Recent wave of AI research has enabled a new brand of synthetic media, called deepfakes.
Deepfakes have impressive photorealism, which has generated exciting new use cases but also raised serious threats to our increasingly digital world.
To mitigate these threats, researchers have tried to come up with new methods for deepfake detection that are more effective than traditional forensics and heavily rely on deep AI technology.
- Score: 21.077064523799677
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: The recent wave of AI research has enabled a new brand of synthetic media,
called deepfakes. Deepfakes have impressive photorealism, which has generated
exciting new use cases but also raised serious threats to our increasingly
digital world. To mitigate these threats, researchers have tried to come up
with new methods for deepfake detection that are more effective than
traditional forensics and heavily rely on deep AI technology. In this paper,
following up on encouraging prior work for deepfake detection with attribution
and ensemble techniques, we explore and compare multiple designs for ensemble
detectors. The goal is to achieve robustness and good generalization ability by
leveraging ensembles of models that specialize in different manipulation
categories. Our results corroborate that ensembles can achieve higher accuracy
than individual models when properly tuned, while the generalization ability
relies on access to a large number of training data for a diverse set of known
manipulations.
Related papers
- Diffusion Deepfake [41.59597965760673]
Recent progress in generative AI, primarily through diffusion models, presents significant challenges for real-world deepfake detection.
The increased realism in image details, diverse content, and widespread accessibility to the general public complicates the identification of these sophisticated deepfakes.
This paper introduces two extensive deepfake datasets generated by state-of-the-art diffusion models.
arXiv Detail & Related papers (2024-04-02T02:17:50Z) - Deepfake Sentry: Harnessing Ensemble Intelligence for Resilient Detection and Generalisation [0.8796261172196743]
We propose a proactive and sustainable deepfake training augmentation solution.
We employ a pool of autoencoders that mimic the effect of the artefacts introduced by the deepfake generator models.
Experiments reveal that our proposed ensemble autoencoder-based data augmentation learning approach offers improvements in terms of generalisation.
arXiv Detail & Related papers (2024-03-29T19:09:08Z) - GenFace: A Large-Scale Fine-Grained Face Forgery Benchmark and Cross
Appearance-Edge Learning [49.93362169016503]
The rapid advancement of photorealistic generators has reached a critical juncture where the discrepancy between authentic and manipulated images is increasingly indistinguishable.
Although there have been a number of publicly available face forgery datasets, the forgery faces are mostly generated using GAN-based synthesis technology.
We propose a large-scale, diverse, and fine-grained high-fidelity dataset, namely GenFace, to facilitate the advancement of deepfake detection.
arXiv Detail & Related papers (2024-02-03T03:13:50Z) - Generalized Deepfakes Detection with Reconstructed-Blended Images and
Multi-scale Feature Reconstruction Network [14.749857283918157]
We present a blended-based detection approach that has robust applicability to unseen datasets.
Experiments demonstrated that this approach results in better performance in both cross-manipulation detection and cross-dataset detection on unseen data.
arXiv Detail & Related papers (2023-12-13T09:49:15Z) - GazeForensics: DeepFake Detection via Gaze-guided Spatial Inconsistency
Learning [63.547321642941974]
We introduce GazeForensics, an innovative DeepFake detection method that utilizes gaze representation obtained from a 3D gaze estimation model.
Experiment results reveal that our proposed GazeForensics outperforms the current state-of-the-art methods.
arXiv Detail & Related papers (2023-11-13T04:48:33Z) - Improving Cross-dataset Deepfake Detection with Deep Information
Decomposition [57.284370468207214]
Deepfake technology poses a significant threat to security and social trust.
Existing detection methods suffer from sharp performance degradation when faced with cross-dataset scenarios.
We propose a deep information decomposition (DID) framework in this paper.
arXiv Detail & Related papers (2023-09-30T12:30:25Z) - SeeABLE: Soft Discrepancies and Bounded Contrastive Learning for
Exposing Deepfakes [7.553507857251396]
We propose a novel deepfake detector, called SeeABLE, that formalizes the detection problem as a (one-class) out-of-distribution detection task.
SeeABLE pushes perturbed faces towards predefined prototypes using a novel regression-based bounded contrastive loss.
We show that our model convincingly outperforms competing state-of-the-art detectors, while exhibiting highly encouraging generalization capabilities.
arXiv Detail & Related papers (2022-11-21T09:38:30Z) - Beyond the Spectrum: Detecting Deepfakes via Re-Synthesis [69.09526348527203]
Deep generative models have led to highly realistic media, known as deepfakes, that are commonly indistinguishable from real to human eyes.
We propose a novel fake detection that is designed to re-synthesize testing images and extract visual cues for detection.
We demonstrate the improved effectiveness, cross-GAN generalization, and robustness against perturbations of our approach in a variety of detection scenarios.
arXiv Detail & Related papers (2021-05-29T21:22:24Z) - TAR: Generalized Forensic Framework to Detect Deepfakes using Weakly
Supervised Learning [17.40885531847159]
Deepfakes have become a critical social problem, and detecting them is of utmost importance.
In this work, we introduce a practical digital forensic tool to detect different types of deepfakes simultaneously.
We develop an autoencoder-based detection model with Residual blocks and sequentially perform transfer learning to detect different types of deepfakes simultaneously.
arXiv Detail & Related papers (2021-05-13T07:31:08Z) - M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection [74.19291916812921]
forged images generated by Deepfake techniques pose a serious threat to the trustworthiness of digital information.
In this paper, we aim to capture the subtle manipulation artifacts at different scales for Deepfake detection.
We introduce a high-quality Deepfake dataset, SR-DF, which consists of 4,000 DeepFake videos generated by state-of-the-art face swapping and facial reenactment methods.
arXiv Detail & Related papers (2021-04-20T05:43:44Z) - Artificial Fingerprinting for Generative Models: Rooting Deepfake
Attribution in Training Data [64.65952078807086]
Photorealistic image generation has reached a new level of quality due to the breakthroughs of generative adversarial networks (GANs)
Yet, the dark side of such deepfakes, the malicious use of generated media, raises concerns about visual misinformation.
We seek a proactive and sustainable solution on deepfake detection by introducing artificial fingerprints into the models.
arXiv Detail & Related papers (2020-07-16T16:49:55Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.