How Robust are LLMs to In-Context Majority Label Bias?
- URL: http://arxiv.org/abs/2312.16549v1
- Date: Wed, 27 Dec 2023 12:20:12 GMT
- Title: How Robust are LLMs to In-Context Majority Label Bias?
- Authors: Karan Gupta, Sumegh Roychowdhury, Siva Rajesh Kasa, Santhosh Kumar
Kasa, Anish Bhanushali, Nikhil Pattisapu, Prasanna Srinivasa Murthy
- Abstract summary: We study the robustness of in-context learning in Large Language Models (LLMs) to shifts that occur due to majority label bias.
Our findings also highlight the impact of model size and the richness of instructional prompts contributing towards model robustness.
- Score: 3.3577727874594654
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: In the In-Context Learning (ICL) setup, various forms of label biases can
manifest. One such manifestation is majority label bias, which arises when the
distribution of labeled examples in the in-context samples is skewed towards
one or more specific classes making Large Language Models (LLMs) more prone to
predict those labels. Such discrepancies can arise from various factors,
including logistical constraints, inherent biases in data collection methods,
limited access to diverse data sources, etc. which are unavoidable in a
real-world industry setup. In this work, we study the robustness of in-context
learning in LLMs to shifts that occur due to majority label bias within the
purview of text classification tasks. Prior works have shown that in-context
learning with LLMs is susceptible to such biases. In our study, we go one level
deeper and show that the robustness boundary varies widely for different models
and tasks, with certain LLMs being highly robust (~90%) to majority label bias.
Additionally, our findings also highlight the impact of model size and the
richness of instructional prompts contributing towards model robustness. We
restrict our study to only publicly available open-source models to ensure
transparency and reproducibility.
Related papers
- Uncertainty Aware Learning for Language Model Alignment [97.36361196793929]
We propose uncertainty-aware learning (UAL) to improve the model alignment of different task scenarios.
We implement UAL in a simple fashion -- adaptively setting the label smoothing value of training according to the uncertainty of individual samples.
Experiments on widely used benchmarks demonstrate that our UAL significantly and consistently outperforms standard supervised fine-tuning.
arXiv Detail & Related papers (2024-06-07T11:37:45Z) - BEADs: Bias Evaluation Across Domains [9.19312529999677]
Large language models (LLMs) can inherit and perpetuate biases from their training data.
We introduce the Bias Evaluations Across Domains (BEADs) dataset to support a wide range of NLP tasks.
Our empirical analysis shows that BEADs is effective in detecting and reducing biases across different language models.
arXiv Detail & Related papers (2024-06-06T16:18:30Z) - Investigating Bias in LLM-Based Bias Detection: Disparities between LLMs and Human Perception [13.592532358127293]
We investigate the presence and nature of bias within Large Language Models (LLMs)
We probe whether LLMs exhibit biases, particularly in political bias prediction and text continuation tasks.
We propose debiasing strategies, including prompt engineering and model fine-tuning.
arXiv Detail & Related papers (2024-03-22T00:59:48Z) - Few-Shot Fairness: Unveiling LLM's Potential for Fairness-Aware
Classification [7.696798306913988]
We introduce a framework outlining fairness regulations aligned with various fairness definitions.
We explore the configuration for in-context learning and the procedure for selecting in-context demonstrations using RAG.
Experiments conducted with different LLMs indicate that GPT-4 delivers superior results in terms of both accuracy and fairness compared to other models.
arXiv Detail & Related papers (2024-02-28T17:29:27Z) - Exploring Value Biases: How LLMs Deviate Towards the Ideal [57.99044181599786]
Large-Language-Models (LLMs) are deployed in a wide range of applications, and their response has an increasing social impact.
We show that value bias is strong in LLMs across different categories, similar to the results found in human studies.
arXiv Detail & Related papers (2024-02-16T18:28:43Z) - GPTBIAS: A Comprehensive Framework for Evaluating Bias in Large Language
Models [83.30078426829627]
Large language models (LLMs) have gained popularity and are being widely adopted by a large user community.
The existing evaluation methods have many constraints, and their results exhibit a limited degree of interpretability.
We propose a bias evaluation framework named GPTBIAS that leverages the high performance of LLMs to assess bias in models.
arXiv Detail & Related papers (2023-12-11T12:02:14Z) - ROBBIE: Robust Bias Evaluation of Large Generative Language Models [27.864027322486375]
Different prompt-based datasets can be used to measure social bias across multiple text domains and demographic axes.
We compare 6 different prompt-based bias and toxicity metrics across 12 demographic axes and 5 families of generative LLMs.
We conduct a comprehensive study of how well 3 bias/toxicity mitigation techniques perform across our suite of measurements.
arXiv Detail & Related papers (2023-11-29T23:03:04Z) - Confronting LLMs with Traditional ML: Rethinking the Fairness of Large Language Models in Tabular Classifications [23.963586791210414]
We show that large language models (LLMs) tend to inherit social biases from their training data which significantly impact their fairness in classification tasks.
This observation emphasizes that the social biases are inherent within the LLMs themselves and inherited from their pretraining corpus.
arXiv Detail & Related papers (2023-10-23T06:31:28Z) - Few-shot Instruction Prompts for Pretrained Language Models to Detect
Social Biases [55.45617404586874]
We propose a few-shot instruction-based method for prompting pre-trained language models (LMs)
We show that large LMs can detect different types of fine-grained biases with similar and sometimes superior accuracy to fine-tuned models.
arXiv Detail & Related papers (2021-12-15T04:19:52Z) - Improving Contrastive Learning on Imbalanced Seed Data via Open-World
Sampling [96.8742582581744]
We present an open-world unlabeled data sampling framework called Model-Aware K-center (MAK)
MAK follows three simple principles: tailness, proximity, and diversity.
We demonstrate that MAK can consistently improve both the overall representation quality and the class balancedness of the learned features.
arXiv Detail & Related papers (2021-11-01T15:09:41Z) - Creating Training Sets via Weak Indirect Supervision [66.77795318313372]
Weak Supervision (WS) frameworks synthesize training labels from multiple potentially noisy supervision sources.
We formulate Weak Indirect Supervision (WIS), a new research problem for automatically synthesizing training labels.
We develop a probabilistic modeling approach, PLRM, which uses user-provided label relations to model and leverage indirect supervision sources.
arXiv Detail & Related papers (2021-10-07T14:09:35Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.