Mitigating Bias in RAG: Controlling the Embedder

• URL: http://arxiv.org/abs/2502.17390v1
• Date: Mon, 24 Feb 2025 18:16:10 GMT
• Title: Mitigating Bias in RAG: Controlling the Embedder
• Abstract summary: We study the conflict between biases of each component and their relationship to the overall bias of the RAG system.<n>We show that bias conflict can be characterized through a linear relationship among components despite its complexity.<n>Our results underscore that a fair RAG system can be better achieved by carefully controlling the bias of the embedder rather than increasing its fairness.
• Score: 26.799581301768622
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In retrieval augmented generation (RAG) systems, each individual component -- the LLM, embedder, and corpus -- could introduce biases in the form of skews towards outputting certain perspectives or identities. In this work, we study the conflict between biases of each component and their relationship to the overall bias of the RAG system, which we call bias conflict. Examining both gender and political biases as case studies, we show that bias conflict can be characterized through a linear relationship among components despite its complexity in 6 different LLMs. Through comprehensive fine-tuning experiments creating 120 differently biased embedders, we demonstrate how to control bias while maintaining utility and reveal the importance of reverse-biasing the embedder to mitigate bias in the overall system. Additionally, we find that LLMs and tasks exhibit varying sensitivities to the embedder bias, a crucial factor to consider for debiasing. Our results underscore that a fair RAG system can be better achieved by carefully controlling the bias of the embedder rather than increasing its fairness.

Related papers

• How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs? [53.58941416780391]
  We study where this susceptibility, spanning sycophancy and related cue-induced biases, lives inside the model.<n>We extract a per-bias direction from hidden states and triangulate it through three measures: probing, leave-one-dataset-out transfer, and causal intervention.
  arXiv  Detail & Related papers  (2026-07-20T16:10:06Z)
• Examining Agents' Bias Amplification versus Suppression in Multi-Agent Systems [5.210390383114933]
  We show that agents endowed with bias can substantially affect system-wide fairness.<n>When agents are exposed to bias uniformly, the system-wide bias elevates, even exceeding the additive sum of the individual agents' biases.
  arXiv  Detail & Related papers  (2026-05-27T07:53:29Z)
• CogBias: Measuring and Mitigating Cognitive Bias in Large Language Models [8.81733256907103]
  Large Language Models (LLMs) are increasingly deployed in high-stakes decision-making contexts.<n>We show that cognitive biases emerge systematically across all four families of cognitive biases: Judgment, Information Processing, Social, and Response.<n>We apply activation steering to modulate biased behavior, achieving 26--32% reduction in bias score.
  arXiv  Detail & Related papers  (2026-04-01T20:22:14Z)
• Evaluating Social Bias in RAG Systems: When External Context Helps and Reasoning Hurts [7.344577590113121]
  Social biases inherent in large language models (LLMs) raise significant fairness concerns.<n>This work focuses on evaluating and understanding the social bias implications of RAG.
  arXiv  Detail & Related papers  (2026-02-10T06:27:56Z)
• Making Bias Non-Predictive: Training Robust LLM Judges via Reinforcement Learning [91.8584139564909]
  Large language models (LLMs) increasingly serve as automated judges, yet they remain susceptible to cognitive biases.<n>We propose Epistemic Independence Training (EIT), a reinforcement learning framework grounded in a key principle.<n>EIT operationalizes this through a balanced conflict strategy where bias signals are equally likely to support correct and incorrect answers.
  arXiv  Detail & Related papers  (2026-02-02T01:43:48Z)
• The Unfairness of Multifactorial Bias in Recommendation [68.35079031029616]
  Popularity bias and positivity bias are prominent sources of bias in recommender systems.<n>In this work, we examine how multifactorial bias influences item-side fairness.<n>We adapt a percentile-based rating transformation as a pre-processing strategy to mitigate multifactorial bias.
  arXiv  Detail & Related papers  (2026-01-19T08:37:43Z)
• Multi Attribute Bias Mitigation via Representation Learning [5.155573439009767]
  Real world images frequently exhibit multiple overlapping biases, including textures, watermarks, gendered makeup, scene object pairings, etc.<n>We tackle this multi bias problem with Generalized Multi Bias Mitigation (GMBM), a lean two stage framework that needs group labels only while training and minimizes bias at test time.<n>We validate GMBM on FB CMNIST, CelebA, and COCO, where we boost worst group accuracy, halve multi attribute bias amplification, and set a new low in SBA.
  arXiv  Detail & Related papers  (2025-09-03T18:08:59Z)
• Who is in the Spotlight: The Hidden Bias Undermining Multimodal Retrieval-Augmented Generation [39.545788636148025]
  We present the first comprehensive study of position bias in multimodal RAG systems.<n>Our results reveal that multimodal interactions intensify position bias compared to unimodal settings.<n>These findings highlight the need for evidence reordering or debiasing strategies to build more reliable and equitable generation systems.
  arXiv  Detail & Related papers  (2025-05-30T06:48:02Z)
• Judging with Many Minds: Do More Perspectives Mean Less Prejudice? On Bias Amplifications and Resistance in Multi-Agent Based LLM-as-Judge [70.89799989428367]
  We conduct a systematic analysis of four diverse bias types: position bias, verbosity bias, chain-of-thought bias, and bandwagon bias.<n>We evaluate these biases across two widely adopted multi-agent LLM-as-Judge frameworks: Multi-Agent-Debate and LLM-as-Meta-Judge.
  arXiv  Detail & Related papers  (2025-05-26T03:56:41Z)
• How far can bias go? -- Tracing bias from pretraining data to alignment [54.51310112013655]
  This study examines the correlation between gender-occupation bias in pre-training data and their manifestation in LLMs.<n>Our findings reveal that biases present in pre-training data are amplified in model outputs.
  arXiv  Detail & Related papers  (2024-11-28T16:20:25Z)
• Investigating Implicit Bias in Large Language Models: A Large-Scale Study of Over 50 LLMs [0.0]
  Large Language Models (LLMs) are being adopted across a wide range of tasks. Recent research indicates that LLMs can harbor implicit biases even when they pass explicit bias evaluations. This study highlights that newer or larger language models do not automatically exhibit reduced bias.
  arXiv  Detail & Related papers  (2024-10-13T03:43:18Z)
• Unveiling and Mitigating Bias in Large Language Model Recommendations: A Path to Fairness [3.5297361401370044]
  This study explores the interplay between bias and LLM-based recommendation systems. It focuses on music, song, and book recommendations across diverse demographic and cultural groups. Our findings reveal that bias in these systems is deeply ingrained, yet even simple interventions like prompt engineering can significantly reduce it.
  arXiv  Detail & Related papers  (2024-09-17T01:37:57Z)
• Editable Fairness: Fine-Grained Bias Mitigation in Language Models [52.66450426729818]
  We propose a novel debiasing approach, Fairness Stamp (FAST), which enables fine-grained calibration of individual social biases. FAST surpasses state-of-the-art baselines with superior debiasing performance. This highlights the potential of fine-grained debiasing strategies to achieve fairness in large language models.
  arXiv  Detail & Related papers  (2024-08-07T17:14:58Z)
• Investigating Bias in LLM-Based Bias Detection: Disparities between LLMs and Human Perception [13.592532358127293]
  We investigate the presence and nature of bias within Large Language Models (LLMs)<n>We probe whether LLMs exhibit biases, particularly in political bias prediction and text continuation tasks.<n>We propose debiasing strategies, including prompt engineering and model fine-tuning.
  arXiv  Detail & Related papers  (2024-03-22T00:59:48Z)
• Bias-Conflict Sample Synthesis and Adversarial Removal Debias Strategy for Temporal Sentence Grounding in Video [67.24316233946381]
  Temporal Sentence Grounding in Video (TSGV) is troubled by dataset bias issue. We propose the bias-conflict sample synthesis and adversarial removal debias strategy (BSSARD)
  arXiv  Detail & Related papers  (2024-01-15T09:59:43Z)
• Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis [86.49858739347412]
  Large Language Models (LLMs) have sparked intense debate regarding the prevalence of bias in these models and its mitigation. We propose a prompt-based method for the extraction of confounding and mediating attributes which contribute to the decision process. We find that the observed disparate treatment can at least in part be attributed to confounding and mitigating attributes and model misalignment.
  arXiv  Detail & Related papers  (2023-11-15T00:02:25Z)
• Step by Step to Fairness: Attributing Societal Bias in Task-oriented Dialogue Systems [42.96878402632241]
  We propose a diagnosis method to attribute bias to each component of a TOD system. We conduct experiments to attribute the TOD system's bias toward three demographic axes: gender, age, and race.
  arXiv  Detail & Related papers  (2023-11-11T09:06:15Z)
• Bias Propagation in Federated Learning [22.954608704251118]
  We show that the bias of a few parties against under-represented groups can propagate through the network to all the parties in the network. We analyze and explain bias propagation in federated learning on naturally partitioned real-world datasets.
  arXiv  Detail & Related papers  (2023-09-05T11:55:03Z)
• Delving into Identify-Emphasize Paradigm for Combating Unknown Bias [52.76758938921129]
  We propose an effective bias-conflicting scoring method (ECS) to boost the identification accuracy. We also propose gradient alignment (GA) to balance the contributions of the mined bias-aligned and bias-conflicting samples. Experiments are conducted on multiple datasets in various settings, demonstrating that the proposed solution can mitigate the impact of unknown biases.
  arXiv  Detail & Related papers  (2023-02-22T14:50:24Z)
• Self-supervised debiasing using low rank regularization [59.84695042540525]
  Spurious correlations can cause strong biases in deep neural networks, impairing generalization ability. We propose a self-supervised debiasing framework potentially compatible with unlabeled samples. Remarkably, the proposed debiasing framework significantly improves the generalization performance of self-supervised learning baselines.
  arXiv  Detail & Related papers  (2022-10-11T08:26:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.