Challenges in Annotating Datasets to Quantify Bias in Under-represented
Society
- URL: http://arxiv.org/abs/2309.08624v1
- Date: Mon, 11 Sep 2023 22:24:39 GMT
- Title: Challenges in Annotating Datasets to Quantify Bias in Under-represented
Society
- Authors: Vithya Yogarajan, Gillian Dobbie, Timothy Pistotti, Joshua Bensemann,
Kobe Knowles
- Abstract summary: Benchmark bias datasets have been developed for binary gender classification and ethical/racial considerations.
Motivated by the lack of annotated datasets for quantifying bias in under-represented societies, we created benchmark datasets for the New Zealand (NZ) population.
This research outlines the manual annotation process, provides an overview of the challenges we encountered and lessons learnt, and presents recommendations for future research.
- Score: 7.9342597513806865
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Recent advances in artificial intelligence, including the development of
highly sophisticated large language models (LLM), have proven beneficial in
many real-world applications. However, evidence of inherent bias encoded in
these LLMs has raised concerns about equity. In response, there has been an
increase in research dealing with bias, including studies focusing on
quantifying bias and developing debiasing techniques. Benchmark bias datasets
have also been developed for binary gender classification and ethical/racial
considerations, focusing predominantly on American demographics. However, there
is minimal research in understanding and quantifying bias related to
under-represented societies. Motivated by the lack of annotated datasets for
quantifying bias in under-represented societies, we endeavoured to create
benchmark datasets for the New Zealand (NZ) population. We faced many
challenges in this process, despite the availability of three annotators. This
research outlines the manual annotation process, provides an overview of the
challenges we encountered and lessons learnt, and presents recommendations for
future research.
Related papers
- Thinking Racial Bias in Fair Forgery Detection: Models, Datasets and Evaluations [63.52709761339949]
We first contribute a dedicated dataset called the Fair Forgery Detection (FairFD) dataset, where we prove the racial bias of public state-of-the-art (SOTA) methods.
Different from existing forgery detection datasets, the self-construct FairFD dataset contains a balanced racial ratio and diverse forgery generation images with the largest-scale subjects.
We design novel metrics including Approach Averaged Metric and Utility Regularized Metric, which can avoid deceptive results.
arXiv Detail & Related papers (2024-07-19T14:53:18Z) - Fairness and Bias in Multimodal AI: A Survey [0.20971479389679337]
We fill a gap with regards to the minimal study of fairness and bias in Large Multimodal Models.
We identify a new category of quantifying bias (preuse)
We critically discuss the various ways researchers are addressing these challenges.
arXiv Detail & Related papers (2024-06-27T11:26:17Z) - VLBiasBench: A Comprehensive Benchmark for Evaluating Bias in Large Vision-Language Model [72.13121434085116]
VLBiasBench is a benchmark aimed at evaluating biases in Large Vision-Language Models (LVLMs)
We construct a dataset encompassing nine distinct categories of social biases, including age, disability status, gender, nationality, physical appearance, race, religion, profession, social economic status and two intersectional bias categories (race x gender, and race x social economic status)
We conduct extensive evaluations on 15 open-source models as well as one advanced closed-source model, providing some new insights into the biases revealing from these models.
arXiv Detail & Related papers (2024-06-20T10:56:59Z) - Tackling Bias in Pre-trained Language Models: Current Trends and
Under-represented Societies [6.831519625084861]
This research presents a survey synthesising the current trends and limitations in techniques used for identifying and mitigating bias in language models.
We argue that current practices tackling the bias problem cannot simply be 'plugged in' to address the needs of under-represented societies.
arXiv Detail & Related papers (2023-12-03T21:25:10Z) - Survey of Social Bias in Vision-Language Models [65.44579542312489]
Survey aims to provide researchers with a high-level insight into the similarities and differences of social bias studies in pre-trained models across NLP, CV, and VL.
The findings and recommendations presented here can benefit the ML community, fostering the development of fairer and non-biased AI models.
arXiv Detail & Related papers (2023-09-24T15:34:56Z) - Bias and Fairness in Large Language Models: A Survey [73.87651986156006]
We present a comprehensive survey of bias evaluation and mitigation techniques for large language models (LLMs)
We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing.
We then unify the literature by proposing three intuitive, two for bias evaluation, and one for mitigation.
arXiv Detail & Related papers (2023-09-02T00:32:55Z) - CBBQ: A Chinese Bias Benchmark Dataset Curated with Human-AI
Collaboration for Large Language Models [52.25049362267279]
We present a Chinese Bias Benchmark dataset that consists of over 100K questions jointly constructed by human experts and generative language models.
The testing instances in the dataset are automatically derived from 3K+ high-quality templates manually authored with stringent quality control.
Extensive experiments demonstrate the effectiveness of the dataset in detecting model bias, with all 10 publicly available Chinese large language models exhibiting strong bias in certain categories.
arXiv Detail & Related papers (2023-06-28T14:14:44Z) - Testing Occupational Gender Bias in Language Models: Towards Robust Measurement and Zero-Shot Debiasing [98.07536837448293]
Large language models (LLMs) have been shown to exhibit a variety of harmful, human-like biases against various demographics.
We introduce a list of desiderata for robustly measuring biases in generative language models.
We then use this benchmark to test several state-of-the-art open-source LLMs, including Llama, Mistral, and their instruction-tuned versions.
arXiv Detail & Related papers (2022-12-20T22:41:24Z) - Assessing Demographic Bias Transfer from Dataset to Model: A Case Study
in Facial Expression Recognition [1.5340540198612824]
Two metrics focus on the representational and stereotypical bias of the dataset, and the third one on the residual bias of the trained model.
We demonstrate the usefulness of the metrics by applying them to a FER problem based on the popular Affectnet dataset.
arXiv Detail & Related papers (2022-05-20T09:40:42Z) - Representation Bias in Data: A Survey on Identification and Resolution
Techniques [26.142021257838564]
Data-driven algorithms are only as good as the data they work with, while data sets, especially social data, often fail to represent minorities adequately.
Representation Bias in data can happen due to various reasons ranging from historical discrimination to selection and sampling biases in the data acquisition and preparation methods.
This paper reviews the literature on identifying and resolving representation bias as a feature of a data set, independent of how consumed later.
arXiv Detail & Related papers (2022-03-22T16:30:22Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.