Robustness Disparities in Commercial Face Detection

• URL: http://arxiv.org/abs/2108.12508v1
• Date: Fri, 27 Aug 2021 21:37:16 GMT
• Title: Robustness Disparities in Commercial Face Detection
• Authors: Samuel Dooley and Tom Goldstein and John P. Dickerson
• Abstract summary: We present the first of its kind detailed benchmark of the robustness of three such systems: Amazon Rekognition, Microsoft Azure, and Google Cloud Platform. We generally find that photos of individuals who are older, masculine presenting, of darker skin type, or have dim lighting are more susceptible to errors than their counterparts in other identities.
• Score: 72.25318723264215
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Facial detection and analysis systems have been deployed by large companies and critiqued by scholars and activists for the past decade. Critiques that focus on system performance analyze disparity of the system's output, i.e., how frequently is a face detected for different Fitzpatrick skin types or perceived genders. However, we focus on the robustness of these system outputs under noisy natural perturbations. We present the first of its kind detailed benchmark of the robustness of three such systems: Amazon Rekognition, Microsoft Azure, and Google Cloud Platform. We use both standard and recently released academic facial datasets to quantitatively analyze trends in robustness for each. Across all the datasets and systems, we generally find that photos of individuals who are older, masculine presenting, of darker skin type, or have dim lighting are more susceptible to errors than their counterparts in other identities.

Related papers

• Synthetic Counterfactual Faces [1.3062016289815055]
  We build a generative AI framework to construct targeted, counterfactual, high-quality synthetic face data. Our pipeline has many use cases, including face recognition systems sensitivity evaluations and image understanding system probes. We showcase the efficacy of our face generation pipeline on a leading commercial vision model.
  arXiv  Detail & Related papers  (2024-07-18T22:22:49Z)
• Stable Bias: Analyzing Societal Representations in Diffusion Models [72.27121528451528]
  We propose a new method for exploring the social biases in Text-to-Image (TTI) systems. Our approach relies on characterizing the variation in generated images triggered by enumerating gender and ethnicity markers in the prompts. We leverage this method to analyze images generated by 3 popular TTI systems and find that while all of their outputs show correlations with US labor demographics, they also consistently under-represent marginalized identities to different extents.
  arXiv  Detail & Related papers  (2023-03-20T19:32:49Z)
• Robustness Disparities in Face Detection [64.71318433419636]
  We present the first of its kind detailed benchmark of face detection systems, specifically examining the robustness to noise of commercial and academic models. Across all the datasets and systems, we generally find that photos of individuals who are $textitmasculine presenting$, of $textitolder$, of $textitdarker skin type$, or have $textitdim lighting$ are more susceptible to errors than their counterparts in other identities.
  arXiv  Detail & Related papers  (2022-11-29T05:22:47Z)
• My Face My Choice: Privacy Enhancing Deepfakes for Social Media Anonymization [4.725675279167593]
  We introduce three face access models in a hypothetical social network, where the user has the power to only appear in photos they approve. Our approach eclipses current tagging systems and replaces unapproved faces with quantitatively dissimilar deepfakes. Running seven SOTA face recognizers on our results, MFMC reduces the average accuracy by 61%.
  arXiv  Detail & Related papers  (2022-11-02T17:58:20Z)
• Are Commercial Face Detection Models as Biased as Academic Models? [64.71318433419636]
  We compare academic and commercial face detection systems, specifically examining robustness to noise. We find that state-of-the-art academic face detection models exhibit demographic disparities in their noise robustness. We conclude that commercial models are always as biased or more biased than an academic model.
  arXiv  Detail & Related papers  (2022-01-25T02:21:42Z)
• Anatomizing Bias in Facial Analysis [86.79402670904338]
  Existing facial analysis systems have been shown to yield biased results against certain demographic subgroups. It has become imperative to ensure that these systems do not discriminate based on gender, identity, or skin tone of individuals. This has led to research in the identification and mitigation of bias in AI systems.
  arXiv  Detail & Related papers  (2021-12-13T09:51:13Z)
• Evaluation of Human and Machine Face Detection using a Novel Distinctive Human Appearance Dataset [0.76146285961466]
  We evaluate current state-of-the-art face-detection models in their ability to detect faces in images. The evaluation results show that face-detection algorithms do not generalize well to diverse appearances.
  arXiv  Detail & Related papers  (2021-11-01T02:20:40Z)
• GenderRobustness: Robustness of Gender Detection in Facial Recognition Systems with variation in Image Properties [0.5330240017302619]
  There have been increasing accusations on artificial intelligence systems and algorithms of computer vision of possessing implicit biases. One such class of systems where bias is said to exist is facial recognition systems, where bias has been observed on the basis of gender, ethnicity, skin tone and other facial attributes. Developers of these systems to ensure that the bias is kept to a bare minimum or ideally non-existent.
  arXiv  Detail & Related papers  (2020-11-18T18:13:23Z)
• On the Robustness of Face Recognition Algorithms Against Attacks and Bias [78.68458616687634]
  Face recognition algorithms have demonstrated very high recognition performance, suggesting suitability for real world applications. Despite the enhanced accuracies, robustness of these algorithms against attacks and bias has been challenged. This paper summarizes different ways in which the robustness of a face recognition algorithm is challenged.
  arXiv  Detail & Related papers  (2020-02-07T18:21:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.