Improving Fairness in Large-Scale Object Recognition by CrowdSourced Demographic Information

• URL: http://arxiv.org/abs/2206.01326v1
• Date: Thu, 2 Jun 2022 22:55:10 GMT
• Title: Improving Fairness in Large-Scale Object Recognition by CrowdSourced Demographic Information
• Authors: Zu Kim and Andr\'e Araujo and Bingyi Cao and Cam Askew and Jack Sim and Mike Green and N'Mah Fodiatu Yilla and Tobias Weyand
• Abstract summary: Representing objects fairly in machine learning datasets will lead to models that are less biased towards a particular culture. We propose a simple and general approach, based on crowdsourcing the demographic composition of the contributors. We present analysis which leads to a much fairer coverage of the world compared to existing datasets.
• Score: 7.968124582214686
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: There has been increasing awareness of ethical issues in machine learning, and fairness has become an important research topic. Most fairness efforts in computer vision have been focused on human sensing applications and preventing discrimination by people's physical attributes such as race, skin color or age by increasing visual representation for particular demographic groups. We argue that ML fairness efforts should extend to object recognition as well. Buildings, artwork, food and clothing are examples of the objects that define human culture. Representing these objects fairly in machine learning datasets will lead to models that are less biased towards a particular culture and more inclusive of different traditions and values. There exist many research datasets for object recognition, but they have not carefully considered which classes should be included, or how much training data should be collected per class. To address this, we propose a simple and general approach, based on crowdsourcing the demographic composition of the contributors: we define fair relevance scores, estimate them, and assign them to each class. We showcase its application to the landmark recognition domain, presenting a detailed analysis and the final fairer landmark rankings. We present analysis which leads to a much fairer coverage of the world compared to existing datasets. The evaluation dataset was used for the 2021 Google Landmark Challenges, which was the first of a kind with an emphasis on fairness in generic object recognition.

Related papers

• Balancing the Scales: Enhancing Fairness in Facial Expression Recognition with Latent Alignment [5.784550537553534]
  This workleverages representation learning based on latent spaces to mitigate bias in facial expression recognition systems. It also enhances a deep learning model's fairness and overall accuracy.
  arXiv  Detail & Related papers  (2024-10-25T10:03:10Z)
• Fairness and Bias Mitigation in Computer Vision: A Survey [61.01658257223365]
  Computer vision systems are increasingly being deployed in high-stakes real-world applications. There is a dire need to ensure that they do not propagate or amplify any discriminatory tendencies in historical or human-curated data. This paper presents a comprehensive survey on fairness that summarizes and sheds light on ongoing trends and successes in the context of computer vision.
  arXiv  Detail & Related papers  (2024-08-05T13:44:22Z)
• Fairness meets Cross-Domain Learning: a new perspective on Models and Metrics [80.07271410743806]
  We study the relationship between cross-domain learning (CD) and model fairness. We introduce a benchmark on face and medical images spanning several demographic groups as well as classification and localization tasks. Our study covers 14 CD approaches alongside three state-of-the-art fairness algorithms and shows how the former can outperform the latter.
  arXiv  Detail & Related papers  (2023-03-25T09:34:05Z)
• ColorSense: A Study on Color Vision in Machine Visual Recognition [57.916512479603064]
  We collect 110,000 non-trivial human annotations of foreground and background color labels from visual recognition benchmarks. We validate the use of our datasets by demonstrating that the level of color discrimination has a dominating effect on the performance of machine perception models. Our findings suggest that object recognition tasks such as classification and localization are susceptible to color vision bias.
  arXiv  Detail & Related papers  (2022-12-16T18:51:41Z)
• Fairness Indicators for Systematic Assessments of Visual Feature Extractors [21.141633753573764]
  We propose three fairness indicators, which aim at quantifying harms and biases of visual systems. Our indicators use existing publicly available datasets collected for fairness evaluations. These indicators are not intended to be a substitute for a thorough analysis of the broader impact of the new computer vision technologies.
  arXiv  Detail & Related papers  (2022-02-15T17:45:33Z)
• A survey on datasets for fairness-aware machine learning [6.962333053044713]
  A large variety of fairness-aware machine learning solutions have been proposed. In this paper, we overview real-world datasets used for fairness-aware machine learning. For a deeper understanding of bias and fairness in the datasets, we investigate the interesting relationships using exploratory analysis.
  arXiv  Detail & Related papers  (2021-10-01T16:54:04Z)
• Towards A Fairer Landmark Recognition Dataset [9.654500155170172]
  We create a landmark recognition dataset with a focus on fair worldwide representation. We start by defining the fair relevance of a landmark to the world population. These relevances are estimated by combining anonymized Google Maps user contribution statistics with the contributors' demographic information.
  arXiv  Detail & Related papers  (2021-08-19T18:42:22Z)
• Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework [83.21732533130846]
  The paper focuses on large in-the-wild databases, i.e., Aff-Wild and Aff-Wild2. It presents the design of two classes of deep neural networks trained with these databases. A novel multi-task and holistic framework is presented which is able to jointly learn and effectively generalize and perform affect recognition.
  arXiv  Detail & Related papers  (2021-03-29T17:36:20Z)
• Through the Data Management Lens: Experimental Analysis and Evaluation of Fair Classification [75.49600684537117]
  Data management research is showing an increasing presence and interest in topics related to data and algorithmic fairness. We contribute a broad analysis of 13 fair classification approaches and additional variants, over their correctness, fairness, efficiency, scalability, and stability. Our analysis highlights novel insights on the impact of different metrics and high-level approach characteristics on different aspects of performance.
  arXiv  Detail & Related papers  (2021-01-18T22:55:40Z)
• Fairness in Semi-supervised Learning: Unlabeled Data Help to Reduce Discrimination [53.3082498402884]
  A growing specter in the rise of machine learning is whether the decisions made by machine learning models are fair. We present a framework of fair semi-supervised learning in the pre-processing phase, including pseudo labeling to predict labels for unlabeled data. A theoretical decomposition analysis of bias, variance and noise highlights the different sources of discrimination and the impact they have on fairness in semi-supervised learning.
  arXiv  Detail & Related papers  (2020-09-25T05:48:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.