What Should Be Balanced in a "Balanced" Face Recognition Dataset?

• URL: http://arxiv.org/abs/2304.09818v2
• Date: Wed, 23 Aug 2023 23:32:49 GMT
• Title: What Should Be Balanced in a "Balanced" Face Recognition Dataset?
• Authors: Haiyu Wu, Kevin W. Bowyer
• Abstract summary: Various face image datasets have been proposed as 'fair' or 'balanced' to assess the accuracy of face recognition algorithms across demographics. It is important to note that the number of identities and images in an evaluation dataset are em not driving factors for 1-to-1 face matching accuracy. We propose a bias-aware toolkit that facilitates creation of cross-demographic evaluation datasets balanced on factors mentioned in this paper.
• Score: 8.820019122897154
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The issue of demographic disparities in face recognition accuracy has attracted increasing attention in recent years. Various face image datasets have been proposed as 'fair' or 'balanced' to assess the accuracy of face recognition algorithms across demographics. These datasets typically balance the number of identities and images across demographics. It is important to note that the number of identities and images in an evaluation dataset are {\em not} driving factors for 1-to-1 face matching accuracy. Moreover, balancing the number of identities and images does not ensure balance in other factors known to impact accuracy, such as head pose, brightness, and image quality. We demonstrate these issues using several recently proposed datasets. To improve the ability to perform less biased evaluations, we propose a bias-aware toolkit that facilitates creation of cross-demographic evaluation datasets balanced on factors mentioned in this paper.

Related papers

• The Impact of Balancing Real and Synthetic Data on Accuracy and Fairness in Face Recognition [10.849598219674132]
  We investigate the impact of demographically balanced authentic and synthetic data, both individually and in combination, on the accuracy and fairness of face recognition models. Our findings emphasize two main points: (i) the increased effectiveness of training data generated by diffusion-based models in enhancing accuracy, whether used alone or combined with subsets of authentic data, and (ii) the minimal impact of incorporating balanced data from pre-trained generative methods on fairness.
  arXiv  Detail & Related papers  (2024-09-04T16:50:48Z)
• Toward Fairer Face Recognition Datasets [69.04239222633795]
  Face recognition and verification are computer vision tasks whose performance has progressed with the introduction of deep representations. Ethical, legal, and technical challenges due to the sensitive character of face data and biases in real training datasets hinder their development. We promote fairness by introducing a demographic attributes balancing mechanism in generated training datasets.
  arXiv  Detail & Related papers  (2024-06-24T12:33:21Z)
• Classes Are Not Equal: An Empirical Study on Image Recognition Fairness [100.36114135663836]
  We experimentally demonstrate that classes are not equal and the fairness issue is prevalent for image classification models across various datasets. Our findings reveal that models tend to exhibit greater prediction biases for classes that are more challenging to recognize. Data augmentation and representation learning algorithms improve overall performance by promoting fairness to some degree in image classification.
  arXiv  Detail & Related papers  (2024-02-28T07:54:50Z)
• Fairness on Synthetic Visual and Thermal Mask Images [1.4524096882720263]
  We study performance and fairness on visual and thermal images and expand the assessment to masked synthetic images. Using the SpeakingFace and Thermal-Mask dataset, we propose a process to assess fairness on real images and show how the same process can be applied to synthetic images.
  arXiv  Detail & Related papers  (2022-09-19T05:04:42Z)
• A Deep Dive into Dataset Imbalance and Bias in Face Identification [49.210042420757894]
  Media portrayals often center imbalance as the main source of bias in automated face recognition systems. Previous studies of data imbalance in FR have focused exclusively on the face verification setting. This work thoroughly explores the effects of each kind of imbalance possible in face identification, and discuss other factors which may impact bias in this setting.
  arXiv  Detail & Related papers  (2022-03-15T20:23:13Z)
• Comparing Human and Machine Bias in Face Recognition [46.170389064229354]
  We release improvements to the LFW and CelebA datasets which will enable future researchers to obtain measurements of algorithmic bias. We also use these new data to develop a series of challenging facial identification and verification questions. We find that both computer models and human survey participants perform significantly better at the verification task.
  arXiv  Detail & Related papers  (2021-10-15T22:26:20Z)
• Unravelling the Effect of Image Distortions for Biased Prediction of Pre-trained Face Recognition Models [86.79402670904338]
  We evaluate the performance of four state-of-the-art deep face recognition models in the presence of image distortions. We have observed that image distortions have a relationship with the performance gap of the model across different subgroups.
  arXiv  Detail & Related papers  (2021-08-14T16:49:05Z)
• Enhancing Facial Data Diversity with Style-based Face Aging [59.984134070735934]
  In particular, face datasets are typically biased in terms of attributes such as gender, age, and race. We propose a novel, generative style-based architecture for data augmentation that captures fine-grained aging patterns. We show that the proposed method outperforms state-of-the-art algorithms for age transfer.
  arXiv  Detail & Related papers  (2020-06-06T21:53:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.