Mask-up: Investigating Biases in Face Re-identification for Masked Faces

• URL: http://arxiv.org/abs/2402.13771v1
• Date: Wed, 21 Feb 2024 12:48:45 GMT
• Title: Mask-up: Investigating Biases in Face Re-identification for Masked Faces
• Authors: Siddharth D Jaiswal, Ankit Kr. Verma, Animesh Mukherjee
• Abstract summary: Face Recognition Systems (FRSs) are now widely distributed and deployed as ML solutions all over the world. Extensive biases have been reported against marginalized groups in these systems and have led to highly discriminatory outcomes. This study shows that developers, lawmakers and users of such services need to rethink the design principles behind FRSs.
• Score: 7.73812434373948
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI based Face Recognition Systems (FRSs) are now widely distributed and deployed as MLaaS solutions all over the world, moreso since the COVID-19 pandemic for tasks ranging from validating individuals' faces while buying SIM cards to surveillance of citizens. Extensive biases have been reported against marginalized groups in these systems and have led to highly discriminatory outcomes. The post-pandemic world has normalized wearing face masks but FRSs have not kept up with the changing times. As a result, these systems are susceptible to mask based face occlusion. In this study, we audit four commercial and nine open-source FRSs for the task of face re-identification between different varieties of masked and unmasked images across five benchmark datasets (total 14,722 images). These simulate a realistic validation/surveillance task as deployed in all major countries around the world. Three of the commercial and five of the open-source FRSs are highly inaccurate; they further perpetuate biases against non-White individuals, with the lowest accuracy being 0%. A survey for the same task with 85 human participants also results in a low accuracy of 40%. Thus a human-in-the-loop moderation in the pipeline does not alleviate the concerns, as has been frequently hypothesized in literature. Our large-scale study shows that developers, lawmakers and users of such services need to rethink the design principles behind FRSs, especially for the task of face re-identification, taking cognizance of observed biases.

Related papers

• Breaking the Global North Stereotype: A Global South-centric Benchmark Dataset for Auditing and Mitigating Biases in Facial Recognition Systems [7.790132091010725]
  We propose a new face dataset composed of 6,579 unique male and female sportspersons from eight countries around the world. More than 50% of the dataset comprises individuals from the Global South countries and is demographically diverse. To aid adversarial audits and robust model training, each image has four adversarial variants, totaling over 40,000 images.
  arXiv  Detail & Related papers  (2024-07-22T17:22:04Z)
• Generalized Face Liveness Detection via De-spoofing Face Generator [58.7043386978171]
  Previous Face Anti-spoofing (FAS) works face the challenge of generalizing in unseen domains. We conduct an Anomalous cue Guided FAS (AG-FAS) method, which leverages real faces for improving model generalization via a De-spoofing Face Generator (DFG) We then propose an Anomalous cue Guided FAS feature extraction Network (AG-Net) to further improve the FAS feature generalization via a cross-attention transformer.
  arXiv  Detail & Related papers  (2024-01-17T06:59:32Z)
• Real Face Foundation Representation Learning for Generalized Deepfake Detection [74.4691295738097]
  The emergence of deepfake technologies has become a matter of social concern as they pose threats to individual privacy and public security. It is almost impossible to collect sufficient representative fake faces, and it is hard for existing detectors to generalize to all types of manipulation. We propose Real Face Foundation Representation Learning (RFFR), which aims to learn a general representation from large-scale real face datasets.
  arXiv  Detail & Related papers  (2023-03-15T08:27:56Z)
• Bias-Aware Face Mask Detection Dataset [11.400704308166805]
  We present a novel face mask detection dataset that contains images posted on Twitter during the pandemic from around the world. Unlike previous datasets, the proposed Bias-Aware Face Mask Detection dataset contains more images from underrepresented race and age groups.
  arXiv  Detail & Related papers  (2022-11-02T15:38:31Z)
• WebFace260M: A Benchmark for Million-Scale Deep Face Recognition [89.39080252029386]
  We contribute a new million-scale recognition benchmark, containing uncurated 4M identities/260M faces (WebFace260M) and cleaned 2M identities/42M faces (WebFace42M) A distributed framework is developed to train face recognition models efficiently without tampering with the performance. The proposed benchmark shows enormous potential on standard, masked and unbiased face recognition scenarios.
  arXiv  Detail & Related papers  (2022-04-21T14:56:53Z)
• Adversarial Mask: Real-World Adversarial Attack Against Face Recognition Models [66.07662074148142]
  We propose a physical adversarial universal perturbation (UAP) against state-of-the-art deep learning-based facial recognition models. In our experiments, we examined the transferability of our adversarial mask to a wide range of deep learning models and datasets. We validated our adversarial mask effectiveness in real-world experiments by printing the adversarial pattern on a fabric medical face mask.
  arXiv  Detail & Related papers  (2021-11-21T08:13:21Z)
• End2End Occluded Face Recognition by Masking Corrupted Features [82.27588990277192]
  State-of-the-art general face recognition models do not generalize well to occluded face images. This paper presents a novel face recognition method that is robust to occlusions based on a single end-to-end deep neural network. Our approach, named FROM (Face Recognition with Occlusion Masks), learns to discover the corrupted features from the deep convolutional neural networks, and clean them by the dynamically learned masks.
  arXiv  Detail & Related papers  (2021-08-21T09:08:41Z)
• Masked Face Recognition Challenge: The WebFace260M Track Report [81.57455766506197]
  Face Bio-metrics under COVID Workshop and Masked Face Recognition Challenge in ICCV 2021. WebFace260M Track aims to push the frontiers of practical MFR. In the first phase of WebFace260M Track, 69 teams (total 833 solutions) participate in the challenge. There are second phase of the challenge till October 1, 2021 and on-going leaderboard.
  arXiv  Detail & Related papers  (2021-08-16T15:51:51Z)
• Multi-Dataset Benchmarks for Masked Identification using Contrastive Representation Learning [0.0]
  COVID-19 pandemic has drastically changed accepted norms globally. Official documents such as passports, driving license and national identity cards are enrolled with fully uncovered face images. In an airport or security checkpoint it is safer to match the unmasked image of the identifying document to the masked person rather than asking them to remove the mask. We propose a contrastive visual representation learning based pre-training workflow which is specialized to masked vs unmasked face matching.
  arXiv  Detail & Related papers  (2021-06-10T08:58:10Z)
• Masked Face Recognition for Secure Authentication [2.429066522170765]
  Masked faces make it difficult to be detected and recognized, thereby threatening to make the in-house datasets invalid. We present an open-source tool, MaskTheFace to mask faces effectively creating a large dataset of masked faces. We report an increase of 38% in the true positive rate for the Facenet system.
  arXiv  Detail & Related papers  (2020-08-25T15:33:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.