MMNet: Multi-Collaboration and Multi-Supervision Network for Sequential Deepfake Detection

• URL: http://arxiv.org/abs/2307.02733v1
• Date: Thu, 6 Jul 2023 02:32:08 GMT
• Title: MMNet: Multi-Collaboration and Multi-Supervision Network for Sequential Deepfake Detection
• Authors: Ruiyang Xia, Decheng Liu, Jie Li, Lin Yuan, Nannan Wang, Xinbo Gao
• Abstract summary: Sequential deepfake detection aims to identify forged facial regions with the correct sequence for recovery. The recovery of forged images requires knowledge of the manipulation model to implement inverse transformations. We propose Multi-Collaboration and Multi-Supervision Network (MMNet) that handles various spatial scales and sequential permutations in forged face images.
• Score: 81.59191603867586
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Advanced manipulation techniques have provided criminals with opportunities to make social panic or gain illicit profits through the generation of deceptive media, such as forged face images. In response, various deepfake detection methods have been proposed to assess image authenticity. Sequential deepfake detection, which is an extension of deepfake detection, aims to identify forged facial regions with the correct sequence for recovery. Nonetheless, due to the different combinations of spatial and sequential manipulations, forged face images exhibit substantial discrepancies that severely impact detection performance. Additionally, the recovery of forged images requires knowledge of the manipulation model to implement inverse transformations, which is difficult to ascertain as relevant techniques are often concealed by attackers. To address these issues, we propose Multi-Collaboration and Multi-Supervision Network (MMNet) that handles various spatial scales and sequential permutations in forged face images and achieve recovery without requiring knowledge of the corresponding manipulation method. Furthermore, existing evaluation metrics only consider detection accuracy at a single inferring step, without accounting for the matching degree with ground-truth under continuous multiple steps. To overcome this limitation, we propose a novel evaluation metric called Complete Sequence Matching (CSM), which considers the detection accuracy at multiple inferring steps, reflecting the ability to detect integrally forged sequences. Extensive experiments on several typical datasets demonstrate that MMNet achieves state-of-the-art detection performance and independent recovery performance.

Related papers

• Generalized Deepfakes Detection with Reconstructed-Blended Images and Multi-scale Feature Reconstruction Network [14.749857283918157]
  We present a blended-based detection approach that has robust applicability to unseen datasets. Experiments demonstrated that this approach results in better performance in both cross-manipulation detection and cross-dataset detection on unseen data.
  arXiv  Detail & Related papers  (2023-12-13T09:49:15Z)
• DeepFidelity: Perceptual Forgery Fidelity Assessment for Deepfake Detection [67.3143177137102]
  Deepfake detection refers to detecting artificially generated or edited faces in images or videos. We propose a novel Deepfake detection framework named DeepFidelity to adaptively distinguish real and fake faces.
  arXiv  Detail & Related papers  (2023-12-07T07:19:45Z)
• COMICS: End-to-end Bi-grained Contrastive Learning for Multi-face Forgery Detection [56.7599217711363]
  Face forgery recognition methods can only process one face at a time. Most face forgery recognition methods can only process one face at a time. We propose COMICS, an end-to-end framework for multi-face forgery detection.
  arXiv  Detail & Related papers  (2023-08-03T03:37:13Z)
• On the Effectiveness of Image Manipulation Detection in the Age of Social Media [9.227950734832447]
  manipulation detection algorithms often rely on the manipulated regions being sufficiently'' different from the rest of the non-tampered regions in the image. We present an in-depth analysis of deep learning-based and learning-free methods, assessing their performance on benchmark datasets. We propose a novel deep learning-based pre-processing technique that accentuates the anomalies present in manipulated regions.
  arXiv  Detail & Related papers  (2023-04-19T04:05:54Z)
• Detecting and Recovering Sequential DeepFake Manipulation [32.34908534582532]
  We propose a novel research problem called Detecting Sequential DeepFake Manipulation (Seq-DeepFake) Unlike the existing deepfake detection task only demanding a binary label prediction, Seq-DeepFake requires correctly predicting a sequential vector of facial manipulation operations. We build a comprehensive benchmark and set up rigorous evaluation protocols and metrics for this new research problem.
  arXiv  Detail & Related papers  (2022-07-05T17:59:33Z)
• ReDFeat: Recoupling Detection and Description for Multimodal Feature Learning [51.07496081296863]
  We recouple independent constraints of detection and description of multimodal feature learning with a mutual weighting strategy. We propose a detector that possesses a large receptive field and is equipped with learnable non-maximum suppression layers. We build a benchmark that contains cross visible, infrared, near-infrared and synthetic aperture radar image pairs for evaluating the performance of features in feature matching and image registration tasks.
  arXiv  Detail & Related papers  (2022-05-16T04:24:22Z)
• Detect and Locate: A Face Anti-Manipulation Approach with Semantic and Noise-level Supervision [67.73180660609844]
  We propose a conceptually simple but effective method to efficiently detect forged faces in an image. The proposed scheme relies on a segmentation map that delivers meaningful high-level semantic information clues about the image. The proposed model achieves state-of-the-art detection accuracy and remarkable localization performance.
  arXiv  Detail & Related papers  (2021-07-13T02:59:31Z)
• M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection [74.19291916812921]
  forged images generated by Deepfake techniques pose a serious threat to the trustworthiness of digital information. In this paper, we aim to capture the subtle manipulation artifacts at different scales for Deepfake detection. We introduce a high-quality Deepfake dataset, SR-DF, which consists of 4,000 DeepFake videos generated by state-of-the-art face swapping and facial reenactment methods.
  arXiv  Detail & Related papers  (2021-04-20T05:43:44Z)
• Fake face detection via adaptive manipulation traces extraction network [9.892936175042939]
  We propose an adaptive manipulation traces extraction network (AMTEN) to suppress image content and highlight manipulation traces. AMTEN exploits an adaptive convolution layer to predict manipulation traces in the image, which are reused in subsequent layers to maximize manipulation artifacts. When detecting fake face images generated by various FIM techniques, AMTENnet achieves an average accuracy up to 98.52%, which outperforms the state-of-the-art works.
  arXiv  Detail & Related papers  (2020-05-11T09:16:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.