Uncovering Latent Biases in Text: Method and Application to Peer Review

• URL: http://arxiv.org/abs/2010.15300v1
• Date: Thu, 29 Oct 2020 01:24:19 GMT
• Title: Uncovering Latent Biases in Text: Method and Application to Peer Review
• Authors: Emaad Manzoor, Nihar B. Shah
• Abstract summary: We introduce a novel framework to quantify bias in text caused by the visibility of subgroup membership indicators. We apply our framework to quantify biases in the text of peer reviews from a reputed machine learning conference.
• Score: 38.726731935235584
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Quantifying systematic disparities in numerical quantities such as employment rates and wages between population subgroups provides compelling evidence for the existence of societal biases. However, biases in the text written for members of different subgroups (such as in recommendation letters for male and non-male candidates), though widely reported anecdotally, remain challenging to quantify. In this work, we introduce a novel framework to quantify bias in text caused by the visibility of subgroup membership indicators. We develop a nonparametric estimation and inference procedure to estimate this bias. We then formalize an identification strategy to causally link the estimated bias to the visibility of subgroup membership indicators, provided observations from time periods both before and after an identity-hiding policy change. We identify an application wherein "ground truth" bias can be inferred to evaluate our framework, instead of relying on synthetic or secondary data. Specifically, we apply our framework to quantify biases in the text of peer reviews from a reputed machine learning conference before and after the conference adopted a double-blind reviewing policy. We show evidence of biases in the review ratings that serves as "ground truth", and show that our proposed framework accurately detects these biases from the review text without having access to the review ratings.

Related papers

• Reducing annotator bias by belief elicitation [3.0040661953201475]
  We propose a simple method for handling bias in annotations without requirements on the number of annotators or instances. We ask annotators about their beliefs of other annotators' judgements of an instance, under the hypothesis that these beliefs may provide more representative labels than judgements. The results indicate that bias, defined as systematic differences between the two groups of annotators, is consistently reduced when asking for beliefs instead of judgements.
  arXiv  Detail & Related papers  (2024-10-21T07:44:01Z)
• Measuring and Addressing Indexical Bias in Information Retrieval [69.7897730778898]
  PAIR framework supports automatic bias audits for ranked documents or entire IR systems. After introducing DUO, we run an extensive evaluation of 8 IR systems on a new corpus of 32k synthetic and 4.7k natural documents. A human behavioral study validates our approach, showing that our bias metric can help predict when and how indexical bias will shift a reader's opinion.
  arXiv  Detail & Related papers  (2024-06-06T17:42:37Z)
• Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extraction [56.17020601803071]
  Recent research shows that pre-trained language models (PLMs) suffer from "prompt bias" in factual knowledge extraction. This paper aims to improve the reliability of existing benchmarks by thoroughly investigating and mitigating prompt bias.
  arXiv  Detail & Related papers  (2024-03-15T02:04:35Z)
• COBIAS: Contextual Reliability in Bias Assessment [14.594920595573038]
  Large Language Models (LLMs) often inherit biases from the web data they are trained on, which contains stereotypes and prejudices. Current methods for evaluating and mitigating these biases rely on bias-benchmark datasets. We introduce a contextual reliability framework, which evaluates model robustness to biased statements by considering the various contexts in which they may appear.
  arXiv  Detail & Related papers  (2024-02-22T10:46:11Z)
• Semantic Properties of cosine based bias scores for word embeddings [48.0753688775574]
  We propose requirements for bias scores to be considered meaningful for quantifying biases. We analyze cosine based scores from the literature with regard to these requirements. We underline these findings with experiments to show that the bias scores' limitations have an impact in the application case.
  arXiv  Detail & Related papers  (2024-01-27T20:31:10Z)
• GPTBIAS: A Comprehensive Framework for Evaluating Bias in Large Language Models [83.30078426829627]
  Large language models (LLMs) have gained popularity and are being widely adopted by a large user community. The existing evaluation methods have many constraints, and their results exhibit a limited degree of interpretability. We propose a bias evaluation framework named GPTBIAS that leverages the high performance of LLMs to assess bias in models.
  arXiv  Detail & Related papers  (2023-12-11T12:02:14Z)
• The SAME score: Improved cosine based bias score for word embeddings [49.75878234192369]
  We introduce SAME, a novel bias score for semantic bias in embeddings. We show that SAME is capable of measuring semantic bias and identify potential causes for social bias in downstream tasks.
  arXiv  Detail & Related papers  (2022-03-28T09:28:13Z)
• Correcting Exposure Bias for Link Recommendation [31.799185352323807]
  Exposure bias can arise when users are systematically underexposed to certain relevant items. We propose estimators that leverage known exposure probabilities to mitigate this bias. Our methods lead to greater diversity in the recommended papers' fields of study.
  arXiv  Detail & Related papers  (2021-06-13T16:51:41Z)
• The Authors Matter: Understanding and Mitigating Implicit Bias in Deep Text Classification [36.361778457307636]
  Deep text classification models can produce biased outcomes for texts written by authors of certain demographic groups. In this paper, we first demonstrate that implicit bias exists in different text classification tasks for different demographic groups. We then build a learning-based interpretation method to deepen our knowledge of implicit bias.
  arXiv  Detail & Related papers  (2021-05-06T16:17:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.