Demographic Fairness in Face Identification: The Watchlist Imbalance Effect

• URL: http://arxiv.org/abs/2106.08049v2
• Date: Wed, 16 Jun 2021 07:45:48 GMT
• Title: Demographic Fairness in Face Identification: The Watchlist Imbalance Effect
• Authors: Pawel Drozdowski, Christian Rathgeb, Christoph Busch
• Abstract summary: "Watchlist imbalance effect" is referred to as "watchlist imbalance effect" It is shown that the database composition has a huge impact on performance differentials in biometric identification systems.
• Score: 18.151605318786437
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, different researchers have found that the gallery composition of a face database can induce performance differentials to facial identification systems in which a probe image is compared against up to all stored reference images to reach a biometric decision. This negative effect is referred to as "watchlist imbalance effect". In this work, we present a method to theoretically estimate said effect for a biometric identification system given its verification performance across demographic groups and the composition of the used gallery. Further, we report results for identification experiments on differently composed demographic subsets, i.e. females and males, of the public academic MORPH database using the open-source ArcFace face recognition system. It is shown that the database composition has a huge impact on performance differentials in biometric identification systems, even if performance differentials are less pronounced in the verification scenario. This study represents the first detailed analysis of the watchlist imbalance effect which is expected to be of high interest for future research in the field of facial recognition.

Related papers

• An Evaluation of Forensic Facial Recognition [16.17759191184531]
  We describe the construction of a large-scale synthetic facial dataset along with a controlled facial forensic lineup. We evaluate the accuracy of two popular neural-based recognition systems. We find that previously reported face recognition accuracies of more than 95% drop to as low as 65% in this more challenging forensic scenario.
  arXiv  Detail & Related papers  (2023-11-10T16:02:46Z)
• What Should Be Balanced in a "Balanced" Face Recognition Dataset? [8.820019122897154]
  Various face image datasets have been proposed as 'fair' or 'balanced' to assess the accuracy of face recognition algorithms across demographics. It is important to note that the number of identities and images in an evaluation dataset are em not driving factors for 1-to-1 face matching accuracy. We propose a bias-aware toolkit that facilitates creation of cross-demographic evaluation datasets balanced on factors mentioned in this paper.
  arXiv  Detail & Related papers  (2023-04-17T22:02:03Z)
• Exploring Causes of Demographic Variations In Face Recognition Accuracy [10.534382915377025]
  We consider accuracy differences as represented by variations in non-mated (impostor) and / or mated (genuine) distributions for 1-to-1 face matching. Possible causes explored include differences in skin tone, face size and shape, imbalance in number of identities and images in the training data, and amount of face visible in the test data.
  arXiv  Detail & Related papers  (2023-04-14T14:50:59Z)
• Fairness on Synthetic Visual and Thermal Mask Images [1.4524096882720263]
  We study performance and fairness on visual and thermal images and expand the assessment to masked synthetic images. Using the SpeakingFace and Thermal-Mask dataset, we propose a process to assess fairness on real images and show how the same process can be applied to synthetic images.
  arXiv  Detail & Related papers  (2022-09-19T05:04:42Z)
• CIAO! A Contrastive Adaptation Mechanism for Non-Universal Facial Expression Recognition [80.07590100872548]
  We propose Contrastive Inhibitory Adaptati On (CIAO), a mechanism that adapts the last layer of facial encoders to depict specific affective characteristics on different datasets. CIAO presents an improvement in facial expression recognition performance over six different datasets with very unique affective representations.
  arXiv  Detail & Related papers  (2022-08-10T15:46:05Z)
• Towards Intrinsic Common Discriminative Features Learning for Face Forgery Detection using Adversarial Learning [59.548960057358435]
  We propose a novel method which utilizes adversarial learning to eliminate the negative effect of different forgery methods and facial identities. Our face forgery detection model learns to extract common discriminative features through eliminating the effect of forgery methods and facial identities.
  arXiv  Detail & Related papers  (2022-07-08T09:23:59Z)
• A Deep Dive into Dataset Imbalance and Bias in Face Identification [49.210042420757894]
  Media portrayals often center imbalance as the main source of bias in automated face recognition systems. Previous studies of data imbalance in FR have focused exclusively on the face verification setting. This work thoroughly explores the effects of each kind of imbalance possible in face identification, and discuss other factors which may impact bias in this setting.
  arXiv  Detail & Related papers  (2022-03-15T20:23:13Z)
• A Synthesis-Based Approach for Thermal-to-Visible Face Verification [105.63410428506536]
  This paper presents an algorithm that achieves state-of-the-art performance on the ARL-VTF and TUFTS multi-spectral face datasets. We also present MILAB-VTF(B), a challenging multi-spectral face dataset composed of paired thermal and visible videos.
  arXiv  Detail & Related papers  (2021-08-21T17:59:56Z)
• On the Applicability of Synthetic Data for Face Recognition [19.095368725147367]
  The use of face images collected as part of border controls is restricted by the European General Data Protection Law to be processed for no other reason than its original purpose. This paper investigates the suitability of synthetic face images generated with StyleGAN and StyleGAN2 to compensate for the urgent lack of publicly available large-scale test data.
  arXiv  Detail & Related papers  (2021-04-06T22:12:30Z)
• Facial Expressions as a Vulnerability in Face Recognition [73.85525896663371]
  This work explores facial expression bias as a security vulnerability of face recognition systems. We present a comprehensive analysis of how facial expression bias impacts the performance of face recognition technologies.
  arXiv  Detail & Related papers  (2020-11-17T18:12:41Z)
• Investigating the Impact of Inclusion in Face Recognition Training Data on Individual Face Identification [93.5538147928669]
  We audit ArcFace, a state-of-the-art, open source face recognition system, in a large-scale face identification experiment with more than one million distractor images. We find a Rank-1 face identification accuracy of 79.71% for individuals present in the model's training data and an accuracy of 75.73% for those not present.
  arXiv  Detail & Related papers  (2020-01-09T15:50:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.