GenFace: A Large-Scale Fine-Grained Face Forgery Benchmark and Cross Appearance-Edge Learning

• URL: http://arxiv.org/abs/2402.02003v4
• Date: Fri, 27 Sep 2024 06:10:01 GMT
• Title: GenFace: A Large-Scale Fine-Grained Face Forgery Benchmark and Cross Appearance-Edge Learning
• Authors: Yaning Zhang, Zitong Yu, Tianyi Wang, Xiaobin Huang, Linlin Shen, Zan Gao, Jianfeng Ren,
• Abstract summary: The rapid advancement of photorealistic generators has reached a critical juncture where the discrepancy between authentic and manipulated images is increasingly indistinguishable. Although there have been a number of publicly available face forgery datasets, the forgery faces are mostly generated using GAN-based synthesis technology. We propose a large-scale, diverse, and fine-grained high-fidelity dataset, namely GenFace, to facilitate the advancement of deepfake detection.
• Score: 50.7702397913573
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rapid advancement of photorealistic generators has reached a critical juncture where the discrepancy between authentic and manipulated images is increasingly indistinguishable. Thus, benchmarking and advancing techniques detecting digital manipulation become an urgent issue. Although there have been a number of publicly available face forgery datasets, the forgery faces are mostly generated using GAN-based synthesis technology, which does not involve the most recent technologies like diffusion. The diversity and quality of images generated by diffusion models have been significantly improved and thus a much more challenging face forgery dataset shall be used to evaluate SOTA forgery detection literature. In this paper, we propose a large-scale, diverse, and fine-grained high-fidelity dataset, namely GenFace, to facilitate the advancement of deepfake detection, which contains a large number of forgery faces generated by advanced generators such as the diffusion-based model and more detailed labels about the manipulation approaches and adopted generators. In addition to evaluating SOTA approaches on our benchmark, we design an innovative cross appearance-edge learning (CAEL) detector to capture multi-grained appearance and edge global representations, and detect discriminative and general forgery traces. Moreover, we devise an appearance-edge cross-attention (AECA) module to explore the various integrations across two domains. Extensive experiment results and visualizations show that our detection model outperforms the state of the arts on different settings like cross-generator, cross-forgery, and cross-dataset evaluations. Code and datasets will be available at \url{https://github.com/Jenine-321/GenFace

Related papers

• Semi-Truths: A Large-Scale Dataset of AI-Augmented Images for Evaluating Robustness of AI-Generated Image detectors [62.63467652611788]
  We introduce SEMI-TRUTHS, featuring 27,600 real images, 223,400 masks, and 1,472,700 AI-augmented images. Each augmented image is accompanied by metadata for standardized and targeted evaluation of detector robustness. Our findings suggest that state-of-the-art detectors exhibit varying sensitivities to the types and degrees of perturbations, data distributions, and augmentation methods used.
  arXiv  Detail & Related papers  (2024-11-12T01:17:27Z)
• MFCLIP: Multi-modal Fine-grained CLIP for Generalizable Diffusion Face Forgery Detection [64.29452783056253]
  The rapid development of photo-realistic face generation methods has raised significant concerns in society and academia. Although existing approaches mainly capture face forgery patterns using image modality, other modalities like fine-grained noises and texts are not fully explored. We propose a novel multi-modal fine-grained CLIP (MFCLIP) model, which mines comprehensive and fine-grained forgery traces across image-noise modalities.
  arXiv  Detail & Related papers  (2024-09-15T13:08:59Z)
• Improving Interpretability and Robustness for the Detection of AI-Generated Images [6.116075037154215]
  We analyze existing state-of-the-art AIGI detection methods based on frozen CLIP embeddings. We show how to interpret them, shedding light on how images produced by various AI generators differ from real ones.
  arXiv  Detail & Related papers  (2024-06-21T10:33:09Z)
• DA-HFNet: Progressive Fine-Grained Forgery Image Detection and Localization Based on Dual Attention [12.36906630199689]
  We construct a DA-HFNet forged image dataset guided by text or image-assisted GAN and Diffusion model. Our goal is to utilize a hierarchical progressive network to capture forged artifacts at different scales for detection and localization.
  arXiv  Detail & Related papers  (2024-06-03T16:13:33Z)
• D$^3$: Scaling Up Deepfake Detection by Learning from Discrepancy [11.239248133240126]
  We seek a step toward a universal deepfake detection system with better generalization and robustness. We propose our Discrepancy Deepfake Detector framework, whose core idea is to learn the universal artifacts from multiple generators. Our framework achieves a 5.3% accuracy improvement in the OOD testing compared to the current SOTA methods while maintaining the ID performance.
  arXiv  Detail & Related papers  (2024-04-06T10:45:02Z)
• Generalized Deepfakes Detection with Reconstructed-Blended Images and Multi-scale Feature Reconstruction Network [14.749857283918157]
  We present a blended-based detection approach that has robust applicability to unseen datasets. Experiments demonstrated that this approach results in better performance in both cross-manipulation detection and cross-dataset detection on unseen data.
  arXiv  Detail & Related papers  (2023-12-13T09:49:15Z)
• X-Transfer: A Transfer Learning-Based Framework for GAN-Generated Fake Image Detection [33.31312811230408]
  misuse of GANs for generating deceptive images, such as face replacement, raises significant security concerns. This paper introduces a novel GAN-generated image detection algorithm called X-Transfer. It enhances transfer learning by utilizing two neural networks that employ interleaved parallel gradient transmission.
  arXiv  Detail & Related papers  (2023-10-07T01:23:49Z)
• Towards General Visual-Linguistic Face Forgery Detection [95.73987327101143]
  Deepfakes are realistic face manipulations that can pose serious threats to security, privacy, and trust. Existing methods mostly treat this task as binary classification, which uses digital labels or mask signals to train the detection model. We propose a novel paradigm named Visual-Linguistic Face Forgery Detection(VLFFD), which uses fine-grained sentence-level prompts as the annotation.
  arXiv  Detail & Related papers  (2023-07-31T10:22:33Z)
• Hierarchical Forgery Classifier On Multi-modality Face Forgery Clues [61.37306431455152]
  We propose a novel Hierarchical Forgery for Multi-modality Face Forgery Detection (HFC-MFFD) The HFC-MFFD learns robust patches-based hybrid representation to enhance forgery authentication in multiple-modality scenarios. The specific hierarchical face forgery is proposed to alleviate the class imbalance problem and further boost detection performance.
  arXiv  Detail & Related papers  (2022-12-30T10:54:29Z)
• Beyond the Spectrum: Detecting Deepfakes via Re-Synthesis [69.09526348527203]
  Deep generative models have led to highly realistic media, known as deepfakes, that are commonly indistinguishable from real to human eyes. We propose a novel fake detection that is designed to re-synthesize testing images and extract visual cues for detection. We demonstrate the improved effectiveness, cross-GAN generalization, and robustness against perturbations of our approach in a variety of detection scenarios.
  arXiv  Detail & Related papers  (2021-05-29T21:22:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.