Looking at the Overlooked: An Analysis on the Word-Overlap Bias in Natural Language Inference

• URL: http://arxiv.org/abs/2211.03862v1
• Date: Mon, 7 Nov 2022 21:02:23 GMT
• Title: Looking at the Overlooked: An Analysis on the Word-Overlap Bias in Natural Language Inference
• Authors: Sara Rajaee, Yadollah Yaghoobzadeh, Mohammad Taher Pilehvar
• Abstract summary: We focus on an overlooked aspect of the overlap bias in NLI models: the reverse word-overlap bias. Current NLI models are highly biased towards the non-entailment label on instances with low overlap. We investigate the reasons for the emergence of the overlap bias and the role of minority examples in its mitigation.
• Score: 20.112129592923246
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: It has been shown that NLI models are usually biased with respect to the word-overlap between premise and hypothesis; they take this feature as a primary cue for predicting the entailment label. In this paper, we focus on an overlooked aspect of the overlap bias in NLI models: the reverse word-overlap bias. Our experimental results demonstrate that current NLI models are highly biased towards the non-entailment label on instances with low overlap, and the existing debiasing methods, which are reportedly successful on existing challenge datasets, are generally ineffective in addressing this category of bias. We investigate the reasons for the emergence of the overlap bias and the role of minority examples in its mitigation. For the former, we find that the word-overlap bias does not stem from pre-training, and for the latter, we observe that in contrast to the accepted assumption, eliminating minority examples does not affect the generalizability of debiasing methods with respect to the overlap bias.

Related papers

• Say My Name: a Model's Bias Discovery Framework [18.887645415907166]
  We introduce Say My Name'' (SaMyNa), the first tool to identify biases within deep models semantically. Unlike existing methods, our approach focuses on biases learned by the model. Our method can disentangle task-related information and proposes itself as a tool to analyze biases.
  arXiv  Detail & Related papers  (2024-08-18T18:50:59Z)
• Projective Methods for Mitigating Gender Bias in Pre-trained Language Models [10.418595661963062]
  Projective methods are fast to implement, use a small number of saved parameters, and make no updates to the existing model parameters. We find that projective methods can be effective at both intrinsic bias and downstream bias mitigation, but that the two outcomes are not necessarily correlated.
  arXiv  Detail & Related papers  (2024-03-27T17:49:31Z)
• Mitigating Bias for Question Answering Models by Tracking Bias Influence [84.66462028537475]
  We propose BMBI, an approach to mitigate the bias of multiple-choice QA models. Based on the intuition that a model would lean to be more biased if it learns from a biased example, we measure the bias level of a query instance. We show that our method could be applied to multiple QA formulations across multiple bias categories.
  arXiv  Detail & Related papers  (2023-10-13T00:49:09Z)
• Feature-Level Debiased Natural Language Understanding [86.8751772146264]
  Existing natural language understanding (NLU) models often rely on dataset biases to achieve high performance on specific datasets. We propose debiasing contrastive learning (DCT) to mitigate biased latent features and neglect the dynamic nature of bias. DCT outperforms state-of-the-art baselines on out-of-distribution datasets while maintaining in-distribution performance.
  arXiv  Detail & Related papers  (2022-12-11T06:16:14Z)
• How Gender Debiasing Affects Internal Model Representations, and Why It Matters [26.993273464725995]
  We show that intrinsic bias is better indicator of debiasing than the standard WEAT metric. Our framework provides a comprehensive perspective on bias in NLP models, which can be applied to deploy NLP systems in a more informed manner.
  arXiv  Detail & Related papers  (2022-04-14T08:54:15Z)
• The SAME score: Improved cosine based bias score for word embeddings [49.75878234192369]
  We introduce SAME, a novel bias score for semantic bias in embeddings. We show that SAME is capable of measuring semantic bias and identify potential causes for social bias in downstream tasks.
  arXiv  Detail & Related papers  (2022-03-28T09:28:13Z)
• Balancing out Bias: Achieving Fairness Through Training Reweighting [58.201275105195485]
  Bias in natural language processing arises from models learning characteristics of the author such as gender and race. Existing methods for mitigating and measuring bias do not directly account for correlations between author demographics and linguistic variables. This paper introduces a very simple but highly effective method for countering bias using instance reweighting.
  arXiv  Detail & Related papers  (2021-09-16T23:40:28Z)
• Towards Debiasing NLU Models from Unknown Biases [70.31427277842239]
  NLU models often exploit biases to achieve high dataset-specific performance without properly learning the intended task. We present a self-debiasing framework that prevents models from mainly utilizing biases without knowing them in advance.
  arXiv  Detail & Related papers  (2020-09-25T15:49:39Z)
• MDR Cluster-Debias: A Nonlinear WordEmbedding Debiasing Pipeline [3.180013942295509]
  Existing methods for debiasing word embeddings often do so only superficially, in that words that are stereotypically associated with a particular gender can still be clustered together in the debiased space. This paper explores why this residual clustering exists, and how it might be addressed. We identify two potential reasons for which residual bias exists and develop a new pipeline, MDR Cluster-Debias, to mitigate this bias.
  arXiv  Detail & Related papers  (2020-06-20T20:03:07Z)
• Mitigating Gender Bias Amplification in Distribution by Posterior Regularization [75.3529537096899]
  We investigate the gender bias amplification issue from the distribution perspective. We propose a bias mitigation approach based on posterior regularization. Our study sheds the light on understanding the bias amplification.
  arXiv  Detail & Related papers  (2020-05-13T11:07:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.