Mitigating Label Biases for In-context Learning

• URL: http://arxiv.org/abs/2305.19148v3
• Date: Fri, 4 Aug 2023 15:43:19 GMT
• Title: Mitigating Label Biases for In-context Learning
• Authors: Yu Fei, Yifan Hou, Zeming Chen, Antoine Bosselut
• Abstract summary: Various design settings for in-context learning (ICL) can bias a model toward a particular prediction without being reflective of an understanding of the task. In this work, we define a typology for three types of label biases in ICL for text classification: vanilla-label bias, context-label bias, and domain-label bias.
• Score: 28.209613730240633
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Various design settings for in-context learning (ICL), such as the choice and order of the in-context examples, can bias a model toward a particular prediction without being reflective of an understanding of the task. While many studies discuss these design choices, there have been few systematic investigations into categorizing them and mitigating their impact. In this work, we define a typology for three types of label biases in ICL for text classification: vanilla-label bias, context-label bias, and domain-label bias (which we conceptualize and detect for the first time). Our analysis demonstrates that prior label bias calibration methods fall short of addressing all three types of biases. Specifically, domain-label bias restricts LLMs to random-level performance on many tasks regardless of the choice of in-context examples. To mitigate the effect of these biases, we propose a simple bias calibration method that estimates a language model's label bias using random in-domain words from the task corpus. After controlling for this estimated bias when making predictions, our novel domain-context calibration significantly improves the ICL performance of GPT-J and GPT-3 on a wide range of tasks. The gain is substantial on tasks with large domain-label bias (up to 37% in Macro-F1). Furthermore, our results generalize to models with different scales, pretraining methods, and manually-designed task instructions, showing the prevalence of label biases in ICL.

Related papers

• Unlabeled Debiasing in Downstream Tasks via Class-wise Low Variance Regularization [13.773597081543185]
  We introduce a novel debiasing regularization technique based on the class-wise variance of embeddings. Our method does not require attribute labels and targets any attribute, thus addressing the shortcomings of existing debiasing methods.
  arXiv  Detail & Related papers  (2024-09-29T03:56:50Z)
• Towards the Mitigation of Confirmation Bias in Semi-supervised Learning: a Debiased Training Perspective [6.164100243945264]
  Semi-supervised learning (SSL) commonly exhibits confirmation bias, where models disproportionately favor certain classes. We introduce TaMatch, a unified framework for debiased training in SSL. We show that TaMatch significantly outperforms existing state-of-the-art methods across a range of challenging image classification tasks.
  arXiv  Detail & Related papers  (2024-09-26T21:50:30Z)
• A Debiased Nearest Neighbors Framework for Multi-Label Text Classification [13.30576550077694]
  We introduce a DEbiased Nearest Neighbors (DENN) framework for Multi-Label Text Classification (MLTC) To address embedding alignment bias, we propose a debiased contrastive learning strategy, enhancing neighbor consistency on label co-occurrence. For confidence estimation bias, we present a debiased confidence estimation strategy, improving the adaptive combination of predictions from $k$NN and inductive binary classifications.
  arXiv  Detail & Related papers  (2024-08-06T14:00:23Z)
• Beyond Performance: Quantifying and Mitigating Label Bias in LLMs [8.77694178599322]
  We evaluate different approaches to quantifying label bias in a model's predictions. Our investigation reveals substantial label bias in models both before and after debiasing attempts. We propose a novel label bias calibration method tailored for few-shot prompting.
  arXiv  Detail & Related papers  (2024-05-04T19:53:03Z)
• Evaluating the Fairness of Discriminative Foundation Models in Computer Vision [51.176061115977774]
  We propose a novel taxonomy for bias evaluation of discriminative foundation models, such as Contrastive Language-Pretraining (CLIP) We then systematically evaluate existing methods for mitigating bias in these models with respect to our taxonomy. Specifically, we evaluate OpenAI's CLIP and OpenCLIP models for key applications, such as zero-shot classification, image retrieval and image captioning.
  arXiv  Detail & Related papers  (2023-10-18T10:32:39Z)
• Mitigating Bias for Question Answering Models by Tracking Bias Influence [84.66462028537475]
  We propose BMBI, an approach to mitigate the bias of multiple-choice QA models. Based on the intuition that a model would lean to be more biased if it learns from a biased example, we measure the bias level of a query instance. We show that our method could be applied to multiple QA formulations across multiple bias categories.
  arXiv  Detail & Related papers  (2023-10-13T00:49:09Z)
• Large Language Models Are Not Robust Multiple Choice Selectors [117.72712117510953]
  Multiple choice questions (MCQs) serve as a common yet important task format in the evaluation of large language models (LLMs) This work shows that modern LLMs are vulnerable to option position changes due to their inherent "selection bias" We propose a label-free, inference-time debiasing method, called PriDe, which separates the model's prior bias for option IDs from the overall prediction distribution.
  arXiv  Detail & Related papers  (2023-09-07T17:44:56Z)
• Less Learn Shortcut: Analyzing and Mitigating Learning of Spurious Feature-Label Correlation [44.319739489968164]
  Deep neural networks often take dataset biases as a shortcut to make decisions rather than understand tasks. In this study, we focus on the spurious correlation between word features and labels that models learn from the biased data distribution. We propose a training strategy Less-Learn-Shortcut (LLS): our strategy quantifies the biased degree of the biased examples and down-weights them accordingly.
  arXiv  Detail & Related papers  (2022-05-25T09:08:35Z)
• Relieving Long-tailed Instance Segmentation via Pairwise Class Balance [85.53585498649252]
  Long-tailed instance segmentation is a challenging task due to the extreme imbalance of training samples among classes. It causes severe biases of the head classes (with majority samples) against the tailed ones. We propose a novel Pairwise Class Balance (PCB) method, built upon a confusion matrix which is updated during training to accumulate the ongoing prediction preferences.
  arXiv  Detail & Related papers  (2022-01-08T07:48:36Z)
• General Greedy De-bias Learning [163.65789778416172]
  We propose a General Greedy De-bias learning framework (GGD), which greedily trains the biased models and the base model like gradient descent in functional space. GGD can learn a more robust base model under the settings of both task-specific biased models with prior knowledge and self-ensemble biased model without prior knowledge.
  arXiv  Detail & Related papers  (2021-12-20T14:47:32Z)
• Balancing out Bias: Achieving Fairness Through Training Reweighting [58.201275105195485]
  Bias in natural language processing arises from models learning characteristics of the author such as gender and race. Existing methods for mitigating and measuring bias do not directly account for correlations between author demographics and linguistic variables. This paper introduces a very simple but highly effective method for countering bias using instance reweighting.
  arXiv  Detail & Related papers  (2021-09-16T23:40:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.