Measuring and Addressing Indexical Bias in Information Retrieval

• URL: http://arxiv.org/abs/2406.04298v1
• Date: Thu, 6 Jun 2024 17:42:37 GMT
• Title: Measuring and Addressing Indexical Bias in Information Retrieval
• Authors: Caleb Ziems, William Held, Jane Dwivedi-Yu, Diyi Yang,
• Abstract summary: PAIR framework supports automatic bias audits for ranked documents or entire IR systems. After introducing DUO, we run an extensive evaluation of 8 IR systems on a new corpus of 32k synthetic and 4.7k natural documents. A human behavioral study validates our approach, showing that our bias metric can help predict when and how indexical bias will shift a reader's opinion.
• Score: 69.7897730778898
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Information Retrieval (IR) systems are designed to deliver relevant content, but traditional systems may not optimize rankings for fairness, neutrality, or the balance of ideas. Consequently, IR can often introduce indexical biases, or biases in the positional order of documents. Although indexical bias can demonstrably affect people's opinion, voting patterns, and other behaviors, these issues remain understudied as the field lacks reliable metrics and procedures for automatically measuring indexical bias. Towards this end, we introduce the PAIR framework, which supports automatic bias audits for ranked documents or entire IR systems. After introducing DUO, the first general-purpose automatic bias metric, we run an extensive evaluation of 8 IR systems on a new corpus of 32k synthetic and 4.7k natural documents, with 4k queries spanning 1.4k controversial issue topics. A human behavioral study validates our approach, showing that our bias metric can help predict when and how indexical bias will shift a reader's opinion.

Related papers

• Eliminating Position Bias of Language Models: A Mechanistic Approach [119.34143323054143]
  Position bias has proven to be a prevalent issue of modern language models (LMs) Our mechanistic analysis attributes the position bias to two components employed in nearly all state-of-the-art LMs: causal attention and relative positional encodings. By eliminating position bias, models achieve better performance and reliability in downstream tasks, including LM-as-a-judge, retrieval-augmented QA, molecule generation, and math reasoning.
  arXiv  Detail & Related papers  (2024-07-01T09:06:57Z)
• Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extraction [56.17020601803071]
  Recent research shows that pre-trained language models (PLMs) suffer from "prompt bias" in factual knowledge extraction. This paper aims to improve the reliability of existing benchmarks by thoroughly investigating and mitigating prompt bias.
  arXiv  Detail & Related papers  (2024-03-15T02:04:35Z)
• Metrics for popularity bias in dynamic recommender systems [0.0]
  Biased recommendations may lead to decisions that can potentially have adverse effects on individuals, sensitive user groups, and society. This paper focuses on quantifying popularity bias that stems directly from the output of RecSys models. Four metrics to quantify popularity bias in RescSys over time in dynamic setting across different sensitive user groups have been proposed.
  arXiv  Detail & Related papers  (2023-10-12T16:15:30Z)
• Investigating Bias with a Synthetic Data Generator: Empirical Evidence and Philosophical Interpretation [66.64736150040093]
  Machine learning applications are becoming increasingly pervasive in our society. Risk is that they will systematically spread the bias embedded in data. We propose to analyze biases by introducing a framework for generating synthetic data with specific types of bias and their combinations.
  arXiv  Detail & Related papers  (2022-09-13T11:18:50Z)
• How Gender Debiasing Affects Internal Model Representations, and Why It Matters [26.993273464725995]
  We show that intrinsic bias is better indicator of debiasing than the standard WEAT metric. Our framework provides a comprehensive perspective on bias in NLP models, which can be applied to deploy NLP systems in a more informed manner.
  arXiv  Detail & Related papers  (2022-04-14T08:54:15Z)
• The SAME score: Improved cosine based bias score for word embeddings [49.75878234192369]
  We introduce SAME, a novel bias score for semantic bias in embeddings. We show that SAME is capable of measuring semantic bias and identify potential causes for social bias in downstream tasks.
  arXiv  Detail & Related papers  (2022-03-28T09:28:13Z)
• Information-Theoretic Bias Reduction via Causal View of Spurious Correlation [71.9123886505321]
  We propose an information-theoretic bias measurement technique through a causal interpretation of spurious correlation. We present a novel debiasing framework against the algorithmic bias, which incorporates a bias regularization loss. The proposed bias measurement and debiasing approaches are validated in diverse realistic scenarios.
  arXiv  Detail & Related papers  (2022-01-10T01:19:31Z)
• Unbiased Pairwise Learning to Rank in Recommender Systems [4.058828240864671]
  Unbiased learning to rank algorithms are appealing candidates and have already been applied in many applications with single categorical labels. We propose a novel unbiased LTR algorithm to tackle the challenges, which innovatively models position bias in the pairwise fashion. Experiment results on public benchmark datasets and internal live traffic show the superior results of the proposed method for both categorical and continuous labels.
  arXiv  Detail & Related papers  (2021-11-25T06:04:59Z)
• Towards Automatic Bias Detection in Knowledge Graphs [5.402498799294428]
  We describe a framework for identifying biases in knowledge graph embeddings, based on numerical bias metrics. We illustrate the framework with three different bias measures on the task of profession prediction. The relations flagged as biased can then be handed to decision makers for judgement upon subsequent debiasing.
  arXiv  Detail & Related papers  (2021-09-19T03:58:25Z)
• Uncovering Latent Biases in Text: Method and Application to Peer Review [38.726731935235584]
  We introduce a novel framework to quantify bias in text caused by the visibility of subgroup membership indicators. We apply our framework to quantify biases in the text of peer reviews from a reputed machine learning conference.
  arXiv  Detail & Related papers  (2020-10-29T01:24:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.