Reducing Selection Bias in Large Language Models

• URL: http://arxiv.org/abs/2402.01740v3
• Date: Sat, 15 Jun 2024 13:23:21 GMT
• Title: Reducing Selection Bias in Large Language Models
• Authors: J. E. Eicher, R. F. Irgolič,
• Abstract summary: Large Language Models (LLMs) are vital in interpreting and executing semantic tasks. This research critically examines these biases and quantifies the effects on a representative list selection task.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) like gpt-3.5-turbo-0613 and claude-instant-1.2 are vital in interpreting and executing semantic tasks. Unfortunately, these models' inherent biases adversely affect their performance Particularly affected is object selection from lists; a fundamental operation in digital navigation and decision-making. This research critically examines these biases and quantifies the effects on a representative list selection task. To explore these biases, we experiment manipulating temperature, list length, object identity, object type, prompt complexity, and model. We isolated and measured the influence of the biases on selection behavior. Our findings show that bias structure is strongly dependent on the model, with object type modulating the magnitude of the effect. With a strong primacy effect, causing the first objects in a list to be disproportionately represented in outputs. The usage of guard rails, a prompt engineering method of ensuring a response structure, increases bias and decreases instruction adherence when to a selection task. The bias is ablated when the guard rail step is separated from the list sampling step, lowering the complexity of each individual task. We provide LLM applications and theoretically suggest that LLMs experience a form of cognitive load that is compensated for with bias.

Related papers

• Exploiting Primacy Effect To Improve Large Language Models [1.03590082373586]
  This study focuses on primacy bias in fine-tuned Large Language Models (LLMs)<n>We first show that fine-tuning amplifies this bias, probably due to exposure to human-like patterns.<n>We strategically leverage this effect by reordering response options based on semantic similarity to the query, without requiring knowledge of the correct answer.
  arXiv  Detail & Related papers  (2025-07-18T14:18:18Z)
• Systematic Bias in Large Language Models: Discrepant Response Patterns in Binary vs. Continuous Judgment Tasks [13.704342633541454]
  Large Language Models (LLMs) are increasingly used in tasks such as psychological text analysis and decision-making in automated systems. This study examines how different response format: binary versus continuous, may systematically influence LLMs' judgments.
  arXiv  Detail & Related papers  (2025-04-28T03:20:55Z)
• How far can bias go? -- Tracing bias from pretraining data to alignment [54.51310112013655]
  This study examines the correlation between gender-occupation bias in pre-training data and their manifestation in LLMs. Our findings reveal that biases present in pre-training data are amplified in model outputs.
  arXiv  Detail & Related papers  (2024-11-28T16:20:25Z)
• Addressing Blind Guessing: Calibration of Selection Bias in Multiple-Choice Question Answering by Video Language Models [16.34646723046073]
  Video Language Models (VLMs) are designed to answer complex video-focused questions. Current benchmarks fail to capture the full reasoning capabilities of VLMs due to selection bias. This study is the first focused investigation of selection bias in video-to-text LLM-powered models.
  arXiv  Detail & Related papers  (2024-10-18T07:52:22Z)
• Aggregation Artifacts in Subjective Tasks Collapse Large Language Models' Posteriors [74.04775677110179]
  In-context Learning (ICL) has become the primary method for performing natural language tasks with Large Language Models (LLMs) In this work, we examine whether this is the result of the aggregation used in corresponding datasets, where trying to combine low-agreement, disparate annotations might lead to annotation artifacts that create detrimental noise in the prompt. Our results indicate that aggregation is a confounding factor in the modeling of subjective tasks, and advocate focusing on modeling individuals instead.
  arXiv  Detail & Related papers  (2024-10-17T17:16:00Z)
• Revisiting SMoE Language Models by Evaluating Inefficiencies with Task Specific Expert Pruning [78.72226641279863]
  Sparse Mixture of Expert (SMoE) models have emerged as a scalable alternative to dense models in language modeling. Our research explores task-specific model pruning to inform decisions about designing SMoE architectures. We introduce an adaptive task-aware pruning technique UNCURL to reduce the number of experts per MoE layer in an offline manner post-training.
  arXiv  Detail & Related papers  (2024-09-02T22:35:03Z)
• Large Language Models are Biased Reinforcement Learners [0.0]
  We show that large language models (LLMs) exhibit behavioral signatures of a relative value bias. Computational cognitive modeling reveals that LLM behavior is well-described by a simple RL algorithm.
  arXiv  Detail & Related papers  (2024-05-19T01:43:52Z)
• Evaluating Generative Language Models in Information Extraction as Subjective Question Correction [49.729908337372436]
  We propose a new evaluation method, SQC-Score. Inspired by the principles in subjective question correction, we propose a new evaluation method, SQC-Score. Results on three information extraction tasks show that SQC-Score is more preferred by human annotators than the baseline metrics.
  arXiv  Detail & Related papers  (2024-04-04T15:36:53Z)
• Debiasing Multimodal Large Language Models [61.6896704217147]
  Large Vision-Language Models (LVLMs) have become indispensable tools in computer vision and natural language processing. Our investigation reveals a noteworthy bias in the generated content, where the output is primarily influenced by the underlying Large Language Models (LLMs) prior to the input image. To rectify these biases and redirect the model's focus toward vision information, we introduce two simple, training-free strategies.
  arXiv  Detail & Related papers  (2024-03-08T12:35:07Z)
• Taxonomy-based CheckList for Large Language Model Evaluation [0.0]
  We introduce human knowledge into natural language interventions and study pre-trained language models' (LMs) behaviors. Inspired by CheckList behavioral testing, we present a checklist-style task that aims to probe and quantify LMs' unethical behaviors.
  arXiv  Detail & Related papers  (2023-12-15T12:58:07Z)
• Debiasing Algorithm through Model Adaptation [5.482673673984126]
  We perform causal analysis to identify problematic model components and discover that mid-upper feed-forward layers are most prone to convey bias. Based on the analysis results, we intervene in the model by applying a linear projection to the weight matrices of these layers. Our titular method, DAMA, significantly decreases bias as measured by diverse metrics while maintaining the model's performance on downstream tasks.
  arXiv  Detail & Related papers  (2023-10-29T05:50:03Z)
• Mind the instructions: a holistic evaluation of consistency and interactions in prompt-based learning [14.569770617709073]
  We present a detailed analysis of which design choices cause instabilities and inconsistencies in task predictions. We show how spurious correlations between input distributions and labels form only a minor problem for prompted models. We statistically analyse the results to show which factors are the most influential, interactive or stable.
  arXiv  Detail & Related papers  (2023-10-20T13:25:24Z)
• Delving into Identify-Emphasize Paradigm for Combating Unknown Bias [52.76758938921129]
  We propose an effective bias-conflicting scoring method (ECS) to boost the identification accuracy. We also propose gradient alignment (GA) to balance the contributions of the mined bias-aligned and bias-conflicting samples. Experiments are conducted on multiple datasets in various settings, demonstrating that the proposed solution can mitigate the impact of unknown biases.
  arXiv  Detail & Related papers  (2023-02-22T14:50:24Z)
• Feature-Level Debiased Natural Language Understanding [86.8751772146264]
  Existing natural language understanding (NLU) models often rely on dataset biases to achieve high performance on specific datasets. We propose debiasing contrastive learning (DCT) to mitigate biased latent features and neglect the dynamic nature of bias. DCT outperforms state-of-the-art baselines on out-of-distribution datasets while maintaining in-distribution performance.
  arXiv  Detail & Related papers  (2022-12-11T06:16:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.