Data Representativeness in Accessibility Datasets: A Meta-Analysis

• URL: http://arxiv.org/abs/2207.08037v1
• Date: Sat, 16 Jul 2022 23:32:19 GMT
• Title: Data Representativeness in Accessibility Datasets: A Meta-Analysis
• Authors: Rie Kamikubo, Lining Wang, Crystal Marte, Amnah Mahmood, Hernisa Kacorri
• Abstract summary: We review datasets sourced by people with disabilities and older adults. We find that accessibility datasets represent diverse ages, but have gender and race representation gaps. We hope our effort expands the space of possibility for greater inclusion of marginalized communities in AI-infused systems.
• Score: 7.6597163467929805
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As data-driven systems are increasingly deployed at scale, ethical concerns have arisen around unfair and discriminatory outcomes for historically marginalized groups that are underrepresented in training data. In response, work around AI fairness and inclusion has called for datasets that are representative of various demographic groups.In this paper, we contribute an analysis of the representativeness of age, gender, and race & ethnicity in accessibility datasets - datasets sourced from people with disabilities and older adults - that can potentially play an important role in mitigating bias for inclusive AI-infused applications. We examine the current state of representation within datasets sourced by people with disabilities by reviewing publicly-available information of 190 datasets, we call these accessibility datasets. We find that accessibility datasets represent diverse ages, but have gender and race representation gaps. Additionally, we investigate how the sensitive and complex nature of demographic variables makes classification difficult and inconsistent (e.g., gender, race & ethnicity), with the source of labeling often unknown. By reflecting on the current challenges and opportunities for representation of disabled data contributors, we hope our effort expands the space of possibility for greater inclusion of marginalized communities in AI-infused systems.

Related papers

• Balancing the Scales: Enhancing Fairness in Facial Expression Recognition with Latent Alignment [5.784550537553534]
  This workleverages representation learning based on latent spaces to mitigate bias in facial expression recognition systems. It also enhances a deep learning model's fairness and overall accuracy.
  arXiv  Detail & Related papers  (2024-10-25T10:03:10Z)
• Toward Fairer Face Recognition Datasets [69.04239222633795]
  Face recognition and verification are computer vision tasks whose performance has progressed with the introduction of deep representations. Ethical, legal, and technical challenges due to the sensitive character of face data and biases in real training datasets hinder their development. We promote fairness by introducing a demographic attributes balancing mechanism in generated training datasets.
  arXiv  Detail & Related papers  (2024-06-24T12:33:21Z)
• Human-Centric Multimodal Machine Learning: Recent Advances and Testbed on AI-based Recruitment [66.91538273487379]
  There is a certain consensus about the need to develop AI applications with a Human-Centric approach. Human-Centric Machine Learning needs to be developed based on four main requirements: (i) utility and social good; (ii) privacy and data ownership; (iii) transparency and accountability; and (iv) fairness in AI-driven decision-making processes. We study how current multimodal algorithms based on heterogeneous sources of information are affected by sensitive elements and inner biases in the data.
  arXiv  Detail & Related papers  (2023-02-13T16:44:44Z)
• Ex-Ante Assessment of Discrimination in Dataset [20.574371560492494]
  Data owners face increasing liability for how the use of their data could harm under-priviliged communities. We propose FORESEE, a FORESt of decision trEEs algorithm, which generates a score that captures how likely an individual's response varies with sensitive attributes.
  arXiv  Detail & Related papers  (2022-08-16T19:28:22Z)
• D-BIAS: A Causality-Based Human-in-the-Loop System for Tackling Algorithmic Bias [57.87117733071416]
  We propose D-BIAS, a visual interactive tool that embodies human-in-the-loop AI approach for auditing and mitigating social biases. A user can detect the presence of bias against a group by identifying unfair causal relationships in the causal network. For each interaction, say weakening/deleting a biased causal edge, the system uses a novel method to simulate a new (debiased) dataset.
  arXiv  Detail & Related papers  (2022-08-10T03:41:48Z)
• Assessing Demographic Bias Transfer from Dataset to Model: A Case Study in Facial Expression Recognition [1.5340540198612824]
  Two metrics focus on the representational and stereotypical bias of the dataset, and the third one on the residual bias of the trained model. We demonstrate the usefulness of the metrics by applying them to a FER problem based on the popular Affectnet dataset.
  arXiv  Detail & Related papers  (2022-05-20T09:40:42Z)
• Demographic-Reliant Algorithmic Fairness: Characterizing the Risks of Demographic Data Collection in the Pursuit of Fairness [0.0]
  We consider calls to collect more data on demographics to enable algorithmic fairness. We show how these techniques largely ignore broader questions of data governance and systemic oppression.
  arXiv  Detail & Related papers  (2022-04-18T04:50:09Z)
• Improving Fairness of AI Systems with Lossless De-biasing [15.039284892391565]
  Mitigating bias in AI systems to increase overall fairness has emerged as an important challenge. We present an information-lossless de-biasing technique that targets the scarcity of data in the disadvantaged group.
  arXiv  Detail & Related papers  (2021-05-10T17:38:38Z)
• Supercharging Imbalanced Data Learning With Energy-based Contrastive Representation Transfer [72.5190560787569]
  In computer vision, learning from long tailed datasets is a recurring theme, especially for natural image datasets. Our proposal posits a meta-distributional scenario, where the data generating mechanism is invariant across the label-conditional feature distributions. This allows us to leverage a causal data inflation procedure to enlarge the representation of minority classes.
  arXiv  Detail & Related papers  (2020-11-25T00:13:11Z)
• Leveraging Administrative Data for Bias Audits: Assessing Disparate Coverage with Mobility Data for COVID-19 Policy [61.60099467888073]
  We show how linking administrative data can enable auditing mobility data for bias. We show that older and non-white voters are less likely to be captured by mobility data. We show that allocating public health resources based on such mobility data could disproportionately harm high-risk elderly and minority groups.
  arXiv  Detail & Related papers  (2020-11-14T02:04:14Z)
• Enhancing Facial Data Diversity with Style-based Face Aging [59.984134070735934]
  In particular, face datasets are typically biased in terms of attributes such as gender, age, and race. We propose a novel, generative style-based architecture for data augmentation that captures fine-grained aging patterns. We show that the proposed method outperforms state-of-the-art algorithms for age transfer.
  arXiv  Detail & Related papers  (2020-06-06T21:53:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.