Judging the Judges: A Systematic Investigation of Position Bias in Pairwise Comparative Assessments by LLMs

• URL: http://arxiv.org/abs/2406.07791v6
• Date: Thu, 31 Oct 2024 23:10:55 GMT
• Title: Judging the Judges: A Systematic Investigation of Position Bias in Pairwise Comparative Assessments by LLMs
• Authors: Lin Shi, Chiyu Ma, Wenhua Liang, Weicheng Ma, Soroush Vosoughi,
• Abstract summary: This study introduces a systematic framework to examine position bias in pairwise comparisons. We conducted experiments with 12 LLM judges across MTBench and DevBench, covering 22 tasks and approximately 40 solution-generating models. Our findings confirm that position bias in capable LLM judges is not due to random chances, along with notable variations observed across judges and tasks.
• Score: 25.300569728792887
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: LLM-as-a-Judge presents a promising alternative to human evaluators across various tasks, but inherent biases, especially position bias - a tendency to favor solutions based on their position in the prompt - have compromised its effectiveness. Our study introduces a systematic framework to examine position bias in pairwise comparisons, focusing on repetition stability, position consistency, and preference fairness. This research significantly contributes to the field by introducing new concepts for understanding position bias and providing a multi-dimensional framework for evaluations. We conducted experiments with 12 LLM judges across MTBench and DevBench, covering 22 tasks and approximately 40 solution-generating models - candidates, resulting in over 100,000 evaluation instances. Our findings confirm that position bias in capable LLM judges is not due to random chances, along with notable variations observed across judges and tasks. Moreover, position bias is weakly influenced by the length of prompt components but significantly impacted by the quality gap between solutions. These insights can help optimize judge model selections, improve benchmark design, and inform future research on debiasing strategies, ultimately enhancing the reliability of LLM judges.

Related papers

• Assessing Judging Bias in Large Reasoning Models: An Empirical Study [99.86300466350013]
  Large Reasoning Models (LRMs) like DeepSeek-R1 and OpenAI-o1 have demonstrated remarkable reasoning capabilities. We present a benchmark comparing judging biases between LLMs and LRMs across both subjective preference-alignment datasets and objective fact-based datasets.
  arXiv  Detail & Related papers  (2025-04-14T07:14:27Z)
• JuStRank: Benchmarking LLM Judges for System Ranking [7.507819077549208]
  We conduct the first large-scale study of LLM judges as system rankers. System scores are generated by aggregating judgment scores over multiple system outputs. Our analysis provides a fine-grained characterization of judge behavior, including their decisiveness and bias.
  arXiv  Detail & Related papers  (2024-12-12T18:51:13Z)
• Diverging Preferences: When do Annotators Disagree and do Models Know? [92.24651142187989]
  We develop a taxonomy of disagreement sources spanning 10 categories across four high-level classes. We find that the majority of disagreements are in opposition with standard reward modeling approaches. We develop methods for identifying diverging preferences to mitigate their influence on evaluation and training.
  arXiv  Detail & Related papers  (2024-10-18T17:32:22Z)
• JudgeBench: A Benchmark for Evaluating LLM-based Judges [61.048125269475854]
  JudgeBench is a benchmark for evaluating LLM-based judges on challenging response pairs spanning knowledge, reasoning, math, and coding. Our comprehensive evaluation on a collection of prompted judges, fine-tuned judges, multi-agent judges, and reward models shows that JudgeBench poses a significantly greater challenge than previous benchmarks.
  arXiv  Detail & Related papers  (2024-10-16T17:58:19Z)
• Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge [84.34545223897578]
  Despite their excellence in many domains, potential issues are under-explored, undermining their reliability and the scope of their utility. We identify 12 key potential biases and propose a new automated bias quantification framework-CALM- which quantifies and analyzes each type of bias in LLM-as-a-Judge. Our work highlights the need for stakeholders to address these issues and remind users to exercise caution in LLM-as-a-Judge applications.
  arXiv  Detail & Related papers  (2024-10-03T17:53:30Z)
• Mitigating the Bias of Large Language Model Evaluation [30.67730115141905]
  We propose systematic research about the bias of LLM-as-a-Judge. For closed-source judge models, we apply calibration to mitigate the significance of superficial quality. For open-source judge models, we propose to mitigate the bias by contrastive training, with curated negative samples that deviate from instruction but present better superficial quality.
  arXiv  Detail & Related papers  (2024-09-25T09:52:44Z)
• Direct Judgement Preference Optimization [66.83088028268318]
  We train large language models (LLMs) as generative judges to evaluate and critique other models' outputs. We employ three approaches to collect the preference pairs for different use cases, each aimed at improving our generative judge from a different perspective. Our model robustly counters inherent biases such as position and length bias, flexibly adapts to any evaluation protocol specified by practitioners, and provides helpful language feedback for improving downstream generator models.
  arXiv  Detail & Related papers  (2024-09-23T02:08:20Z)
• From Calculation to Adjudication: Examining LLM judges on Mathematical Reasoning Tasks [11.01213914485374]
  We study large language models (LLMs) on mathematical reasoning tasks. Our analysis uncovers a strong correlation between judgment performance and the candidate model task performance. We show that it is possible to use statistics, such as the task performances of the individual models, to predict judgment performance.
  arXiv  Detail & Related papers  (2024-09-06T10:09:41Z)
• Systematic Evaluation of LLM-as-a-Judge in LLM Alignment Tasks: Explainable Metrics and Diverse Prompt Templates [10.091146498861333]
  Commercial large language models (LLMs) like GPT-4 have been recently employed to evaluate and compare different alignment approaches. We develop a framework to evaluate, compare, and visualize the reliability and alignment of LLM judges.
  arXiv  Detail & Related papers  (2024-08-23T11:49:01Z)
• CEB: Compositional Evaluation Benchmark for Fairness in Large Language Models [58.57987316300529]
  Large Language Models (LLMs) are increasingly deployed to handle various natural language processing (NLP) tasks. To evaluate the biases exhibited by LLMs, researchers have recently proposed a variety of datasets. We propose CEB, a Compositional Evaluation Benchmark that covers different types of bias across different social groups and tasks.
  arXiv  Detail & Related papers  (2024-07-02T16:31:37Z)
• Eliminating Position Bias of Language Models: A Mechanistic Approach [119.34143323054143]
  Position bias has proven to be a prevalent issue of modern language models (LMs) Our mechanistic analysis attributes the position bias to two components employed in nearly all state-of-the-art LMs: causal attention and relative positional encodings. By eliminating position bias, models achieve better performance and reliability in downstream tasks, including LM-as-a-judge, retrieval-augmented QA, molecule generation, and math reasoning.
  arXiv  Detail & Related papers  (2024-07-01T09:06:57Z)
• Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators [48.54465599914978]
  Large Language Models (LLMs) have demonstrated promising capabilities in assessing the quality of generated natural language. LLMs still exhibit biases in evaluation and often struggle to generate coherent evaluations that align with human assessments. We introduce Pairwise-preference Search (PairS), an uncertainty-guided search method that employs LLMs to conduct pairwise comparisons and efficiently ranks candidate texts.
  arXiv  Detail & Related papers  (2024-03-25T17:11:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.