Thinking Racial Bias in Fair Forgery Detection: Models, Datasets and Evaluations

• URL: http://arxiv.org/abs/2407.14367v2
• Date: Sat, 31 Aug 2024 15:28:20 GMT
• Title: Thinking Racial Bias in Fair Forgery Detection: Models, Datasets and Evaluations
• Authors: Decheng Liu, Zongqi Wang, Chunlei Peng, Nannan Wang, Ruimin Hu, Xinbo Gao,
• Abstract summary: We first contribute a dedicated dataset called the Fair Forgery Detection (FairFD) dataset, where we prove the racial bias of public state-of-the-art (SOTA) methods. We design novel metrics including Approach Averaged Metric and Utility Regularized Metric, which can avoid deceptive results. We also present an effective and robust post-processing technique, Bias Pruning with Fair Activations (BPFA), which improves fairness without requiring retraining or weight updates.
• Score: 63.52709761339949
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Due to the successful development of deep image generation technology, forgery detection plays a more important role in social and economic security. Racial bias has not been explored thoroughly in the deep forgery detection field. In the paper, we first contribute a dedicated dataset called the Fair Forgery Detection (FairFD) dataset, where we prove the racial bias of public state-of-the-art (SOTA) methods. Different from existing forgery detection datasets, the self-constructed FairFD dataset contains a balanced racial ratio and diverse forgery generation images with the largest-scale subjects. Additionally, we identify the problems with naive fairness metrics when benchmarking forgery detection models. To comprehensively evaluate fairness, we design novel metrics including Approach Averaged Metric and Utility Regularized Metric, which can avoid deceptive results. We also present an effective and robust post-processing technique, Bias Pruning with Fair Activations (BPFA), which improves fairness without requiring retraining or weight updates. Extensive experiments conducted with 12 representative forgery detection models demonstrate the value of the proposed dataset and the reasonability of the designed fairness metrics. By applying the BPFA to the existing fairest detector, we achieve a new SOTA. Furthermore, we conduct more in-depth analyses to offer more insights to inspire researchers in the community.

Related papers

• Fairness without Sensitive Attributes via Knowledge Sharing [13.141672574114597]
  We propose a confidence-based hierarchical classifier structure called "Reckoner" for reliable fair model learning under the assumption of missing sensitive attributes. Our experimental results show that Reckoner consistently outperforms state-of-the-art baselines in COMPAS dataset and New Adult dataset.
  arXiv  Detail & Related papers  (2024-09-27T06:16:14Z)
• Enhancing Fairness in Unsupervised Graph Anomaly Detection through Disentanglement [33.565252991113766]
  Graph anomaly detection (GAD) is increasingly crucial in various applications, ranging from financial fraud detection to fake news detection. Current GAD methods largely overlook the fairness problem, which might result in discriminatory decisions skewed toward certain demographic groups. We devise a novel DisEntangle-based FairnEss-aware aNomaly Detection framework on the attributed graph, named DEFEND. Our empirical evaluations on real-world datasets reveal that DEFEND performs effectively in GAD and significantly enhances fairness compared to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-06-03T04:48:45Z)
• Fast Model Debias with Machine Unlearning [54.32026474971696]
  Deep neural networks might behave in a biased manner in many real-world scenarios. Existing debiasing methods suffer from high costs in bias labeling or model re-training. We propose a fast model debiasing framework (FMD) which offers an efficient approach to identify, evaluate and remove biases.
  arXiv  Detail & Related papers  (2023-10-19T08:10:57Z)
• Data AUDIT: Identifying Attribute Utility- and Detectability-Induced Bias in Task Models [8.420252576694583]
  We present a first technique for the rigorous, quantitative screening of medical image datasets. Our method decomposes the risks associated with dataset attributes in terms of their detectability and utility. Using our method, we show our screening method reliably identifies nearly imperceptible bias-inducing artifacts.
  arXiv  Detail & Related papers  (2023-04-06T16:50:15Z)
• Delving into Identify-Emphasize Paradigm for Combating Unknown Bias [52.76758938921129]
  We propose an effective bias-conflicting scoring method (ECS) to boost the identification accuracy. We also propose gradient alignment (GA) to balance the contributions of the mined bias-aligned and bias-conflicting samples. Experiments are conducted on multiple datasets in various settings, demonstrating that the proposed solution can mitigate the impact of unknown biases.
  arXiv  Detail & Related papers  (2023-02-22T14:50:24Z)
• Diffusion Denoising Process for Perceptron Bias in Out-of-distribution Detection [67.49587673594276]
  We introduce a new perceptron bias assumption that suggests discriminator models are more sensitive to certain features of the input, leading to the overconfidence problem. We demonstrate that the diffusion denoising process (DDP) of DMs serves as a novel form of asymmetric, which is well-suited to enhance the input and mitigate the overconfidence problem. Our experiments on CIFAR10, CIFAR100, and ImageNet show that our method outperforms SOTA approaches.
  arXiv  Detail & Related papers  (2022-11-21T08:45:08Z)
• Simultaneous Improvement of ML Model Fairness and Performance by Identifying Bias in Data [1.76179873429447]
  We propose a data preprocessing technique that can detect instances ascribing a specific kind of bias that should be removed from the dataset before training. In particular, we claim that in the problem settings where instances exist with similar feature but different labels caused by variation in protected attributes, an inherent bias gets induced in the dataset.
  arXiv  Detail & Related papers  (2022-10-24T13:04:07Z)
• D-BIAS: A Causality-Based Human-in-the-Loop System for Tackling Algorithmic Bias [57.87117733071416]
  We propose D-BIAS, a visual interactive tool that embodies human-in-the-loop AI approach for auditing and mitigating social biases. A user can detect the presence of bias against a group by identifying unfair causal relationships in the causal network. For each interaction, say weakening/deleting a biased causal edge, the system uses a novel method to simulate a new (debiased) dataset.
  arXiv  Detail & Related papers  (2022-08-10T03:41:48Z)
• General Greedy De-bias Learning [163.65789778416172]
  We propose a General Greedy De-bias learning framework (GGD), which greedily trains the biased models and the base model like gradient descent in functional space. GGD can learn a more robust base model under the settings of both task-specific biased models with prior knowledge and self-ensemble biased model without prior knowledge.
  arXiv  Detail & Related papers  (2021-12-20T14:47:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.