Mitigating Bias in Facial Recognition Systems: Centroid Fairness Loss Optimization

• URL: http://arxiv.org/abs/2504.19370v1
• Date: Sun, 27 Apr 2025 22:17:44 GMT
• Title: Mitigating Bias in Facial Recognition Systems: Centroid Fairness Loss Optimization
• Authors: Jean-Rémy Conti, Stéphan Clémençon,
• Abstract summary: societal demand for fair AI systems has put pressure on the research community to develop predictive models that meet new fairness criteria.<n>In particular, the variability of the errors made by certain Facial Recognition (FR) systems across specific segments of the population compromises the deployment of the latter.<n>We propose a novel post-processing approach to improve the fairness of pre-trained FR models by optimizing a regression loss which acts on centroid-based scores.
• Score: 9.537960917804993
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The urging societal demand for fair AI systems has put pressure on the research community to develop predictive models that are not only globally accurate but also meet new fairness criteria, reflecting the lack of disparate mistreatment with respect to sensitive attributes ($\textit{e.g.}$ gender, ethnicity, age). In particular, the variability of the errors made by certain Facial Recognition (FR) systems across specific segments of the population compromises the deployment of the latter, and was judged unacceptable by regulatory authorities. Designing fair FR systems is a very challenging problem, mainly due to the complex and functional nature of the performance measure used in this domain ($\textit{i.e.}$ ROC curves) and because of the huge heterogeneity of the face image datasets usually available for training. In this paper, we propose a novel post-processing approach to improve the fairness of pre-trained FR models by optimizing a regression loss which acts on centroid-based scores. Beyond the computational advantages of the method, we present numerical experiments providing strong empirical evidence of the gain in fairness and of the ability to preserve global accuracy.

Related papers

• FAIR-SIGHT: Fairness Assurance in Image Recognition via Simultaneous Conformal Thresholding and Dynamic Output Repair [4.825037489691159]
  We introduce a post-hoc framework designed to ensure fairness in computer vision systems by combining conformal prediction with a dynamic output repair mechanism.<n>Our approach calculates a fairness-aware non-conformity score that simultaneously assesses prediction errors and fairness violations.<n>When the non-conformity score for a new image exceeds the threshold, FAIR-SIGHT implements targeted corrective adjustments, such as logit shifts for classification and confidence recalibration for detection.
  arXiv  Detail & Related papers  (2025-04-10T02:23:06Z)
• Fair Bilevel Neural Network (FairBiNN): On Balancing fairness and accuracy via Stackelberg Equilibrium [0.3350491650545292]
  Current methods for mitigating bias often result in information loss and an inadequate balance between accuracy and fairness. We propose a novel methodology grounded in bilevel optimization principles. Our deep learning-based approach concurrently optimize for both accuracy and fairness objectives.
  arXiv  Detail & Related papers  (2024-10-21T18:53:39Z)
• Improving Bias in Facial Attribute Classification: A Combined Impact of KL Divergence induced Loss Function and Dual Attention [3.5527561584422465]
  Earlier systems often exhibited demographic bias, particularly in gender and racial classification, with lower accuracy for women and individuals with darker skin tones. This paper presents a method using a dual attention mechanism with a pre-trained Inception-ResNet V1 model, enhanced by KL-divergence regularization and a cross-entropy loss function. The experimental results show significant improvements in both fairness and classification accuracy, providing promising advances in addressing bias and enhancing the reliability of facial recognition systems.
  arXiv  Detail & Related papers  (2024-10-15T01:29:09Z)
• Identifying and Mitigating Social Bias Knowledge in Language Models [52.52955281662332]
  We propose a novel debiasing approach, Fairness Stamp (FAST), which enables fine-grained calibration of individual social biases.<n>FAST surpasses state-of-the-art baselines with superior debiasing performance.<n>This highlights the potential of fine-grained debiasing strategies to achieve fairness in large language models.
  arXiv  Detail & Related papers  (2024-08-07T17:14:58Z)
• Thinking Racial Bias in Fair Forgery Detection: Models, Datasets and Evaluations [63.52709761339949]
  We first contribute a dedicated dataset called the Fair Forgery Detection (FairFD) dataset, where we prove the racial bias of public state-of-the-art (SOTA) methods.<n>We design novel metrics including Approach Averaged Metric and Utility Regularized Metric, which can avoid deceptive results.<n>We also present an effective and robust post-processing technique, Bias Pruning with Fair Activations (BPFA), which improves fairness without requiring retraining or weight updates.
  arXiv  Detail & Related papers  (2024-07-19T14:53:18Z)
• Learning Fairer Representations with FairVIC [0.0]
  Mitigating bias in automated decision-making systems is a critical challenge due to nuanced definitions of fairness and dataset-specific biases.<n>We introduce FairVIC, an innovative approach that enhances fairness in neural networks by integrating variance, invariance, and covariance terms into the loss function during training.<n>We evaluate FairVIC against comparable bias mitigation techniques on benchmark datasets, considering both group and individual fairness, and conduct an ablation study on the accuracy-fairness trade-off.
  arXiv  Detail & Related papers  (2024-04-28T10:10:21Z)
• Learning for Counterfactual Fairness from Observational Data [62.43249746968616]
  Fairness-aware machine learning aims to eliminate biases of learning models against certain subgroups described by certain protected (sensitive) attributes such as race, gender, and age. A prerequisite for existing methods to achieve counterfactual fairness is the prior human knowledge of the causal model for the data. In this work, we address the problem of counterfactually fair prediction from observational data without given causal models by proposing a novel framework CLAIRE.
  arXiv  Detail & Related papers  (2023-07-17T04:08:29Z)
• Chasing Fairness Under Distribution Shift: A Model Weight Perturbation Approach [72.19525160912943]
  We first theoretically demonstrate the inherent connection between distribution shift, data perturbation, and model weight perturbation. We then analyze the sufficient conditions to guarantee fairness for the target dataset. Motivated by these sufficient conditions, we propose robust fairness regularization (RFR)
  arXiv  Detail & Related papers  (2023-03-06T17:19:23Z)
• Stochastic Methods for AUC Optimization subject to AUC-based Fairness Constraints [51.12047280149546]
  A direct approach for obtaining a fair predictive model is to train the model through optimizing its prediction performance subject to fairness constraints. We formulate the training problem of a fairness-aware machine learning model as an AUC optimization problem subject to a class of AUC-based fairness constraints. We demonstrate the effectiveness of our approach on real-world data under different fairness metrics.
  arXiv  Detail & Related papers  (2022-12-23T22:29:08Z)
• Fair and Optimal Classification via Post-Processing [10.163721748735801]
  This paper provides a complete characterization of the inherent tradeoff of demographic parity on classification problems. We show that the minimum error rate achievable by randomized and attribute-aware fair classifiers is given by the optimal value of a Wasserstein-barycenter problem.
  arXiv  Detail & Related papers  (2022-11-03T00:04:04Z)
• Fairness without the sensitive attribute via Causal Variational Autoencoder [17.675997789073907]
  Due to privacy purposes and var-ious regulations such as RGPD in EU, many personal sensitive attributes are frequently not collected. By leveraging recent developments for approximate inference, we propose an approach to fill this gap. Based on a causal graph, we rely on a new variational auto-encoding based framework named SRCVAE to infer a sensitive information proxy.
  arXiv  Detail & Related papers  (2021-09-10T17:12:52Z)
• Domain-Incremental Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognition [5.478764356647437]
  We propose the novel usage of Continual Learning (CL) as a potent bias mitigation method to enhance the fairness of FER systems. We compare different non-CL-based and CL-based methods for their classification accuracy and fairness scores on expression recognition and Action Unit (AU) detection tasks. Our experimental results show that CL-based methods, on average, outperform other popular bias mitigation techniques on both accuracy and fairness metrics.
  arXiv  Detail & Related papers  (2021-03-15T18:22:17Z)
• Supercharging Imbalanced Data Learning With Energy-based Contrastive Representation Transfer [72.5190560787569]
  In computer vision, learning from long tailed datasets is a recurring theme, especially for natural image datasets. Our proposal posits a meta-distributional scenario, where the data generating mechanism is invariant across the label-conditional feature distributions. This allows us to leverage a causal data inflation procedure to enlarge the representation of minority classes.
  arXiv  Detail & Related papers  (2020-11-25T00:13:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.