GBDF: Gender Balanced DeepFake Dataset Towards Fair DeepFake Detection
- URL: http://arxiv.org/abs/2207.10246v1
- Date: Thu, 21 Jul 2022 01:00:40 GMT
- Title: GBDF: Gender Balanced DeepFake Dataset Towards Fair DeepFake Detection
- Authors: Aakash Varma Nadimpalli and Ajita Rattani
- Abstract summary: Facial forgery by deepfakes has raised severe societal concerns.
Recent studies have demonstrated that facial analysis-based deep learning models can discriminate based on protected attributes.
It is vital to evaluate and understand the fairness of deepfake detectors across demographic variations such as gender and race.
- Score: 1.0152838128195467
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Facial forgery by deepfakes has raised severe societal concerns. Several
solutions have been proposed by the vision community to effectively combat the
misinformation on the internet via automated deepfake detection systems. Recent
studies have demonstrated that facial analysis-based deep learning models can
discriminate based on protected attributes. For the commercial adoption and
massive roll-out of the deepfake detection technology, it is vital to evaluate
and understand the fairness (the absence of any prejudice or favoritism) of
deepfake detectors across demographic variations such as gender and race. As
the performance differential of deepfake detectors between demographic
subgroups would impact millions of people of the deprived sub-group. This paper
aims to evaluate the fairness of the deepfake detectors across males and
females. However, existing deepfake datasets are not annotated with demographic
labels to facilitate fairness analysis. To this aim, we manually annotated
existing popular deepfake datasets with gender labels and evaluated the
performance differential of current deepfake detectors across gender. Our
analysis on the gender-labeled version of the datasets suggests (a) current
deepfake datasets have skewed distribution across gender, and (b) commonly
adopted deepfake detectors obtain unequal performance across gender with mostly
males outperforming females. Finally, we contributed a gender-balanced and
annotated deepfake dataset, GBDF, to mitigate the performance differential and
to promote research and development towards fairness-aware deep fake detectors.
The GBDF dataset is publicly available at: https://github.com/aakash4305/GBDF
Related papers
- DF40: Toward Next-Generation Deepfake Detection [62.073997142001424]
Existing works identify top-notch detection algorithms and models by adhering to the common practice: training detectors on one specific dataset (e.g., FF++) and testing them on other prevalent deepfake datasets.
But can these stand-out "winners" be truly applied to tackle the myriad of realistic and diverse deepfakes lurking in the real world?
We construct a highly diverse and large-scale deepfake dataset called DF40, which comprises 40 distinct deepfake techniques.
We then conduct comprehensive evaluations using 4 standard evaluation protocols and 7 representative detectors, resulting in over 2,000 evaluations.
arXiv Detail & Related papers (2024-06-19T12:35:02Z) - Preserving Fairness Generalization in Deepfake Detection [14.485069525871504]
Deepfake detection models can result in unfair performance disparities among demographic groups, such as race and gender.
We propose the first method to address the fairness generalization problem in deepfake detection by simultaneously considering features, loss, and optimization aspects.
Our method employs disentanglement learning to extract demographic and domain-agnostic features, fusing them to encourage fair learning across a flattened loss landscape.
arXiv Detail & Related papers (2024-02-27T05:47:33Z) - Improving Fairness in Deepfake Detection [38.999205139257164]
biases in the data used to train deepfake detectors can lead to disparities in detection accuracy across different races and genders.
We propose novel loss functions that handle both the setting where demographic information is available as well as the case where this information is absent.
arXiv Detail & Related papers (2023-06-29T02:19:49Z) - Analyzing Fairness in Deepfake Detection With Massively Annotated
Databases [9.407035514709293]
We investigate factors causing biased detection in public Deepfake datasets.
We create large-scale demographic and non-demographic annotations with 47 different attributes for five popular Deepfake datasets.
We analyse attributes resulting in AI-bias of three state-of-the-art Deepfake detection backbone models on these datasets.
arXiv Detail & Related papers (2022-08-11T14:28:21Z) - Voice-Face Homogeneity Tells Deepfake [56.334968246631725]
Existing detection approaches contribute to exploring the specific artifacts in deepfake videos.
We propose to perform the deepfake detection from an unexplored voice-face matching view.
Our model obtains significantly improved performance as compared to other state-of-the-art competitors.
arXiv Detail & Related papers (2022-03-04T09:08:50Z) - An Examination of Fairness of AI Models for Deepfake Detection [5.4852920337961235]
We evaluate bias present in deepfake datasets and detection models across protected subgroups.
Using facial datasets balanced by race and gender, we examine three popular deepfake detectors and find large disparities in predictive performances across races.
arXiv Detail & Related papers (2021-05-02T21:55:04Z) - Balancing Biases and Preserving Privacy on Balanced Faces in the Wild [50.915684171879036]
There are demographic biases present in current facial recognition (FR) models.
We introduce our Balanced Faces in the Wild dataset to measure these biases across different ethnic and gender subgroups.
We find that relying on a single score threshold to differentiate between genuine and imposters sample pairs leads to suboptimal results.
We propose a novel domain adaptation learning scheme that uses facial features extracted from state-of-the-art neural networks.
arXiv Detail & Related papers (2021-03-16T15:05:49Z) - WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection [82.42495493102805]
We introduce a new dataset WildDeepfake which consists of 7,314 face sequences extracted from 707 deepfake videos collected completely from the internet.
We conduct a systematic evaluation of a set of baseline detection networks on both existing and our WildDeepfake datasets, and show that WildDeepfake is indeed a more challenging dataset, where the detection performance can decrease drastically.
arXiv Detail & Related papers (2021-01-05T11:10:32Z) - Gender Stereotype Reinforcement: Measuring the Gender Bias Conveyed by
Ranking Algorithms [68.85295025020942]
We propose the Gender Stereotype Reinforcement (GSR) measure, which quantifies the tendency of a Search Engines to support gender stereotypes.
GSR is the first specifically tailored measure for Information Retrieval, capable of quantifying representational harms.
arXiv Detail & Related papers (2020-09-02T20:45:04Z) - Towards Gender-Neutral Face Descriptors for Mitigating Bias in Face
Recognition [51.856693288834975]
State-of-the-art deep networks implicitly encode gender information while being trained for face recognition.
Gender is often viewed as an important attribute with respect to identifying faces.
We present a novel Adversarial Gender De-biasing algorithm (AGENDA)' to reduce the gender information present in face descriptors.
arXiv Detail & Related papers (2020-06-14T08:54:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.