Improving Fairness in Deepfake Detection

• URL: http://arxiv.org/abs/2306.16635v3
• Date: Wed, 8 Nov 2023 17:10:49 GMT
• Title: Improving Fairness in Deepfake Detection
• Authors: Yan Ju, Shu Hu, Shan Jia, George H. Chen, Siwei Lyu
• Abstract summary: biases in the data used to train deepfake detectors can lead to disparities in detection accuracy across different races and genders. We propose novel loss functions that handle both the setting where demographic information is available as well as the case where this information is absent.
• Score: 38.999205139257164
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the development of effective deepfake detectors in recent years, recent studies have demonstrated that biases in the data used to train these detectors can lead to disparities in detection accuracy across different races and genders. This can result in different groups being unfairly targeted or excluded from detection, allowing undetected deepfakes to manipulate public opinion and erode trust in a deepfake detection model. While existing studies have focused on evaluating fairness of deepfake detectors, to the best of our knowledge, no method has been developed to encourage fairness in deepfake detection at the algorithm level. In this work, we make the first attempt to improve deepfake detection fairness by proposing novel loss functions that handle both the setting where demographic information (eg, annotations of race and gender) is available as well as the case where this information is absent. Fundamentally, both approaches can be used to convert many existing deepfake detectors into ones that encourages fairness. Extensive experiments on four deepfake datasets and five deepfake detectors demonstrate the effectiveness and flexibility of our approach in improving deepfake detection fairness. Our code is available at https://github.com/littlejuyan/DF_Fairness.

Related papers

• DF40: Toward Next-Generation Deepfake Detection [62.073997142001424]
  existing works identify top-notch detection algorithms and models by adhering to the common practice: training detectors on one specific dataset and testing them on other prevalent deepfake datasets. But can these stand-out "winners" be truly applied to tackle the myriad of realistic and diverse deepfakes lurking in the real world? We construct a highly diverse deepfake detection dataset called DF40, which comprises 40 distinct deepfake techniques.
  arXiv  Detail & Related papers  (2024-06-19T12:35:02Z)
• Preserving Fairness Generalization in Deepfake Detection [14.485069525871504]
  Deepfake detection models can result in unfair performance disparities among demographic groups, such as race and gender. We propose the first method to address the fairness generalization problem in deepfake detection by simultaneously considering features, loss, and optimization aspects. Our method employs disentanglement learning to extract demographic and domain-agnostic features, fusing them to encourage fair learning across a flattened loss landscape.
  arXiv  Detail & Related papers  (2024-02-27T05:47:33Z)
• Facial Forgery-based Deepfake Detection using Fine-Grained Features [7.378937711027777]
  Facial forgery by deepfakes has caused major security risks and raised severe societal concerns. We formulate deepfake detection as a fine-grained classification problem and propose a new fine-grained solution to it. Our method is based on learning subtle and generalizable features by effectively suppressing background noise and learning discriminative features at various scales for deepfake detection.
  arXiv  Detail & Related papers  (2023-10-10T21:30:05Z)
• CrossDF: Improving Cross-Domain Deepfake Detection with Deep Information Decomposition [53.860796916196634]
  We propose a Deep Information Decomposition (DID) framework to enhance the performance of Cross-dataset Deepfake Detection (CrossDF) Unlike most existing deepfake detection methods, our framework prioritizes high-level semantic features over specific visual artifacts. It adaptively decomposes facial features into deepfake-related and irrelevant information, only using the intrinsic deepfake-related information for real/fake discrimination.
  arXiv  Detail & Related papers  (2023-09-30T12:30:25Z)
• How Generalizable are Deepfake Image Detectors? An Empirical Study [4.42204674141385]
  We present the first empirical study on the generalizability of deepfake detectors. Our study utilizes six deepfake datasets, five deepfake image detection methods, and two model augmentation approaches. We find that detectors are learning unwanted properties specific to synthesis methods and struggling to extract discriminative features.
  arXiv  Detail & Related papers  (2023-08-08T10:30:34Z)
• DeepfakeBench: A Comprehensive Benchmark of Deepfake Detection [55.70982767084996]
  A critical yet frequently overlooked challenge in the field of deepfake detection is the lack of a standardized, unified, comprehensive benchmark. We present the first comprehensive benchmark for deepfake detection, called DeepfakeBench, which offers three key contributions. DeepfakeBench contains 15 state-of-the-art detection methods, 9CL datasets, a series of deepfake detection evaluation protocols and analysis tools, as well as comprehensive evaluations.
  arXiv  Detail & Related papers  (2023-07-04T01:34:41Z)
• Fooling State-of-the-Art Deepfake Detection with High-Quality Deepfakes [2.0883760606514934]
  We show that deepfake detectors proven to generalize well on multiple research datasets still struggle in real-world scenarios with well-crafted fakes. We propose a novel autoencoder for face swapping alongside an advanced face blending technique, which we utilize to generate 90 high-quality deepfakes.
  arXiv  Detail & Related papers  (2023-05-09T09:08:49Z)
• Voice-Face Homogeneity Tells Deepfake [56.334968246631725]
  Existing detection approaches contribute to exploring the specific artifacts in deepfake videos. We propose to perform the deepfake detection from an unexplored voice-face matching view. Our model obtains significantly improved performance as compared to other state-of-the-art competitors.
  arXiv  Detail & Related papers  (2022-03-04T09:08:50Z)
• WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection [82.42495493102805]
  We introduce a new dataset WildDeepfake which consists of 7,314 face sequences extracted from 707 deepfake videos collected completely from the internet. We conduct a systematic evaluation of a set of baseline detection networks on both existing and our WildDeepfake datasets, and show that WildDeepfake is indeed a more challenging dataset, where the detection performance can decrease drastically.
  arXiv  Detail & Related papers  (2021-01-05T11:10:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.