Exploring Disparity-Accuracy Trade-offs in Face Recognition Systems: The Role of Datasets, Architectures, and Loss Functions

• URL: http://arxiv.org/abs/2503.14138v1
• Date: Tue, 18 Mar 2025 11:04:57 GMT
• Title: Exploring Disparity-Accuracy Trade-offs in Face Recognition Systems: The Role of Datasets, Architectures, and Loss Functions
• Authors: Siddharth D Jaiswal, Sagnik Basu, Sandipan Sikdar, Animesh Mukherjee,
• Abstract summary: Face Recognition Systems (FRSs) are deployed worldwide for identity verification and facial attribute analysis.<n>The performance of these models is determined by a complex interdependence among the model architecture, optimization/loss function and datasets.<n>In this work, we perform an in-depth analysis of three FRSs for the task of gender prediction.<n>Our results show that all three components have an individual as well as a combined impact on both accuracy and disparity.
• Score: 6.966858605501567
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automated Face Recognition Systems (FRSs), developed using deep learning models, are deployed worldwide for identity verification and facial attribute analysis. The performance of these models is determined by a complex interdependence among the model architecture, optimization/loss function and datasets. Although FRSs have surpassed human-level accuracy, they continue to be disparate against certain demographics. Due to the ubiquity of applications, it is extremely important to understand the impact of the three components -- model architecture, loss function and face image dataset on the accuracy-disparity trade-off to design better, unbiased platforms. In this work, we perform an in-depth analysis of three FRSs for the task of gender prediction, with various architectural modifications resulting in ten deep-learning models coupled with four loss functions and benchmark them on seven face datasets across 266 evaluation configurations. Our results show that all three components have an individual as well as a combined impact on both accuracy and disparity. We identify that datasets have an inherent property that causes them to perform similarly across models, independent of the choice of loss functions. Moreover, the choice of dataset determines the model's perceived bias -- the same model reports bias in opposite directions for three gender-balanced datasets of ``in-the-wild'' face images of popular individuals. Studying the facial embeddings shows that the models are unable to generalize a uniform definition of what constitutes a ``female face'' as opposed to a ``male face'', due to dataset diversity. We provide recommendations to model developers on using our study as a blueprint for model development and subsequent deployment.

Related papers

• Biased Heritage: How Datasets Shape Models in Facial Expression Recognition [13.77824359359967]
  We study bias propagation from datasets to trained models in image-based Facial Expression Recognition systems.<n>We introduce new bias metrics specifically designed for multiclass problems with multiple demographic groups.<n>Our findings suggest that preventing emotion-specific demographic patterns should be prioritized over general demographic balance in FER datasets.
  arXiv  Detail & Related papers  (2025-03-05T12:25:22Z)
• Faces of Fairness: Examining Bias in Facial Expression Recognition Datasets and Models [2.8893654860442872]
  This study investigates bias sources in FER datasets and models.<n>Four common FER datasets--AffectNet, ExpW, Fer2013, and RAF-DB--are analyzed.<n>This research evaluates the bias and fairness of six deep models, including three state-of-the-art convolutional neural network (CNN) models.
  arXiv  Detail & Related papers  (2025-02-16T09:23:16Z)
• Can foundation models actively gather information in interactive environments to test hypotheses? [56.651636971591536]
  We introduce a framework in which a model must determine the factors influencing a hidden reward function. We investigate whether approaches such as self- throughput and increased inference time improve information gathering efficiency.
  arXiv  Detail & Related papers  (2024-12-09T12:27:21Z)
• Leveraging Synthetic Data for Generalizable and Fair Facial Action Unit Detection [9.404202619102943]
  We propose to use synthetically generated data and multi-source domain adaptation (MSDA) to address the problems of the scarcity of labeled data and the diversity of subjects. Specifically, we propose to generate a diverse dataset through synthetic facial expression re-targeting. To further improve gender fairness, PM2 matches the features of the real data with a female and a male synthetic image.
  arXiv  Detail & Related papers  (2024-03-15T23:50:18Z)
• Towards Personalized Federated Learning via Heterogeneous Model Reassembly [84.44268421053043]
  pFedHR is a framework that leverages heterogeneous model reassembly to achieve personalized federated learning. pFedHR dynamically generates diverse personalized models in an automated manner.
  arXiv  Detail & Related papers  (2023-08-16T19:36:01Z)
• Unifying Flow, Stereo and Depth Estimation [121.54066319299261]
  We present a unified formulation and model for three motion and 3D perception tasks. We formulate all three tasks as a unified dense correspondence matching problem. Our model naturally enables cross-task transfer since the model architecture and parameters are shared across tasks.
  arXiv  Detail & Related papers  (2022-11-10T18:59:54Z)
• Rethinking Bias Mitigation: Fairer Architectures Make for Fairer Face Recognition [107.58227666024791]
  Face recognition systems are widely deployed in safety-critical applications, including law enforcement. They exhibit bias across a range of socio-demographic dimensions, such as gender and race. Previous works on bias mitigation largely focused on pre-processing the training data.
  arXiv  Detail & Related papers  (2022-10-18T15:46:05Z)
• Towards Explaining Demographic Bias through the Eyes of Face Recognition Models [6.889667606945215]
  Biases inherent in both data and algorithms make the fairness of machine learning (ML)-based decision-making systems less than optimal. We aim at providing a set of explainability tool that analyse the difference in the face recognition models' behaviors when processing different demographic groups. We do that by leveraging higher-order statistical information based on activation maps to build explainability tools that link the FR models' behavior differences to certain facial regions.
  arXiv  Detail & Related papers  (2022-08-29T07:23:06Z)
• Assessing Demographic Bias Transfer from Dataset to Model: A Case Study in Facial Expression Recognition [1.5340540198612824]
  Two metrics focus on the representational and stereotypical bias of the dataset, and the third one on the residual bias of the trained model. We demonstrate the usefulness of the metrics by applying them to a FER problem based on the popular Affectnet dataset.
  arXiv  Detail & Related papers  (2022-05-20T09:40:42Z)
• Taxonomizing local versus global structure in neural network loss landscapes [60.206524503782006]
  We show that the best test accuracy is obtained when the loss landscape is globally well-connected. We also show that globally poorly-connected landscapes can arise when models are small or when they are trained to lower quality data.
  arXiv  Detail & Related papers  (2021-07-23T13:37:14Z)
• Dynamic Federated Learning [57.14673504239551]
  Federated learning has emerged as an umbrella term for centralized coordination strategies in multi-agent environments. We consider a federated learning model where at every iteration, a random subset of available agents perform local updates based on their data. Under a non-stationary random walk model on the true minimizer for the aggregate optimization problem, we establish that the performance of the architecture is determined by three factors, namely, the data variability at each agent, the model variability across all agents, and a tracking term that is inversely proportional to the learning rate of the algorithm.
  arXiv  Detail & Related papers  (2020-02-20T15:00:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.