Biased Heritage: How Datasets Shape Models in Facial Expression Recognition

• URL: http://arxiv.org/abs/2503.03446v1
• Date: Wed, 05 Mar 2025 12:25:22 GMT
• Title: Biased Heritage: How Datasets Shape Models in Facial Expression Recognition
• Authors: Iris Dominguez-Catena, Daniel Paternain, Mikel Galar, MaryBeth Defrance, Maarten Buyl, Tijl De Bie,
• Abstract summary: We study bias propagation from datasets to trained models in image-based Facial Expression Recognition systems.<n>We introduce new bias metrics specifically designed for multiclass problems with multiple demographic groups.<n>Our findings suggest that preventing emotion-specific demographic patterns should be prioritized over general demographic balance in FER datasets.
• Score: 13.77824359359967
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, the rapid development of artificial intelligence (AI) systems has raised concerns about our ability to ensure their fairness, that is, how to avoid discrimination based on protected characteristics such as gender, race, or age. While algorithmic fairness is well-studied in simple binary classification tasks on tabular data, its application to complex, real-world scenarios-such as Facial Expression Recognition (FER)-remains underexplored. FER presents unique challenges: it is inherently multiclass, and biases emerge across intersecting demographic variables, each potentially comprising multiple protected groups. We present a comprehensive framework to analyze bias propagation from datasets to trained models in image-based FER systems, while introducing new bias metrics specifically designed for multiclass problems with multiple demographic groups. Our methodology studies bias propagation by (1) inducing controlled biases in FER datasets, (2) training models on these biased datasets, and (3) analyzing the correlation between dataset bias metrics and model fairness notions. Our findings reveal that stereotypical biases propagate more strongly to model predictions than representational biases, suggesting that preventing emotion-specific demographic patterns should be prioritized over general demographic balance in FER datasets. Additionally, we observe that biased datasets lead to reduced model accuracy, challenging the assumed fairness-accuracy trade-off.

Related papers

• Identifying and Mitigating Social Bias Knowledge in Language Models [52.52955281662332]
  We propose a novel debiasing approach, Fairness Stamp (FAST), which enables fine-grained calibration of individual social biases. FAST surpasses state-of-the-art baselines with superior debiasing performance. This highlights the potential of fine-grained debiasing strategies to achieve fairness in large language models.
  arXiv  Detail & Related papers  (2024-08-07T17:14:58Z)
• Dataset Scale and Societal Consistency Mediate Facial Impression Bias in Vision-Language AI [17.101569078791492]
  We study 43 CLIP vision-language models to determine whether they learn human-like facial impression biases. We show for the first time that the the degree to which a bias is shared across a society predicts the degree to which it is reflected in a CLIP model.
  arXiv  Detail & Related papers  (2024-08-04T08:26:58Z)
• Less can be more: representational vs. stereotypical gender bias in facial expression recognition [3.9698529891342207]
  Machine learning models can inherit biases from their training data, leading to discriminatory or inaccurate predictions. This paper investigates the propagation of demographic biases from datasets into machine learning models. We focus on the gender demographic component, analyzing two types of bias: representational and stereotypical.
  arXiv  Detail & Related papers  (2024-06-25T09:26:49Z)
• Fast Model Debias with Machine Unlearning [54.32026474971696]
  Deep neural networks might behave in a biased manner in many real-world scenarios. Existing debiasing methods suffer from high costs in bias labeling or model re-training. We propose a fast model debiasing framework (FMD) which offers an efficient approach to identify, evaluate and remove biases.
  arXiv  Detail & Related papers  (2023-10-19T08:10:57Z)
• Causality and Independence Enhancement for Biased Node Classification [56.38828085943763]
  We propose a novel Causality and Independence Enhancement (CIE) framework, applicable to various graph neural networks (GNNs) Our approach estimates causal and spurious features at the node representation level and mitigates the influence of spurious correlations. Our approach CIE not only significantly enhances the performance of GNNs but outperforms state-of-the-art debiased node classification methods.
  arXiv  Detail & Related papers  (2023-10-14T13:56:24Z)
• Learning for Counterfactual Fairness from Observational Data [62.43249746968616]
  Fairness-aware machine learning aims to eliminate biases of learning models against certain subgroups described by certain protected (sensitive) attributes such as race, gender, and age. A prerequisite for existing methods to achieve counterfactual fairness is the prior human knowledge of the causal model for the data. In this work, we address the problem of counterfactually fair prediction from observational data without given causal models by proposing a novel framework CLAIRE.
  arXiv  Detail & Related papers  (2023-07-17T04:08:29Z)
• Rethinking Bias Mitigation: Fairer Architectures Make for Fairer Face Recognition [107.58227666024791]
  Face recognition systems are widely deployed in safety-critical applications, including law enforcement. They exhibit bias across a range of socio-demographic dimensions, such as gender and race. Previous works on bias mitigation largely focused on pre-processing the training data.
  arXiv  Detail & Related papers  (2022-10-18T15:46:05Z)
• Gender Stereotyping Impact in Facial Expression Recognition [1.5340540198612824]
  In recent years, machine learning-based models have become the most popular approach to Facial Expression Recognition (FER) In publicly available FER datasets, apparent gender representation is usually mostly balanced, but their representation in the individual label is not. We generate derivative datasets with different amounts of stereotypical bias by altering the gender proportions of certain labels. We observe a discrepancy in the recognition of certain emotions between genders of up to $29 %$ under the worst bias conditions.
  arXiv  Detail & Related papers  (2022-10-11T10:52:23Z)
• Assessing Demographic Bias Transfer from Dataset to Model: A Case Study in Facial Expression Recognition [1.5340540198612824]
  Two metrics focus on the representational and stereotypical bias of the dataset, and the third one on the residual bias of the trained model. We demonstrate the usefulness of the metrics by applying them to a FER problem based on the popular Affectnet dataset.
  arXiv  Detail & Related papers  (2022-05-20T09:40:42Z)
• General Greedy De-bias Learning [163.65789778416172]
  We propose a General Greedy De-bias learning framework (GGD), which greedily trains the biased models and the base model like gradient descent in functional space. GGD can learn a more robust base model under the settings of both task-specific biased models with prior knowledge and self-ensemble biased model without prior knowledge.
  arXiv  Detail & Related papers  (2021-12-20T14:47:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.