Evaluating Scoring Bias in LLM-as-a-Judge

• URL: http://arxiv.org/abs/2506.22316v1
• Date: Fri, 27 Jun 2025 15:25:23 GMT
• Title: Evaluating Scoring Bias in LLM-as-a-Judge
• Authors: Qingquan Li, Shaoyu Dou, Kailai Shao, Chao Chen, Haixiang Hu,
• Abstract summary: Large Language Models (LLMs) are employed as evaluators for complex tasks.<n>There are various biases within LLM-as-a-Judge, which adversely affect the fairness and reliability of judgments.
• Score: 8.751901240110888
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The remarkable performance of Large Language Models (LLMs) gives rise to``LLM-as-a-Judge'', where LLMs are employed as evaluators for complex tasks. Moreover, it has been widely adopted across fields such as Natural Language Processing (NLP), preference learning, and various specific domains. However, there are various biases within LLM-as-a-Judge, which adversely affect the fairness and reliability of judgments. Current research on evaluating or mitigating bias in LLM-as-a-Judge predominantly focuses on comparison-based evaluations, while systematic investigations into bias in scoring-based evaluations remain limited. Therefore, we define scoring bias in LLM-as-a-Judge as the scores differ when scoring judge models are bias-related perturbed, and provide a well-designed framework to comprehensively evaluate scoring bias. We augment existing LLM-as-a-Judge benchmarks through data synthesis to construct our evaluation dataset and design multi-faceted evaluation metrics. Our experimental results demonstrate that the scoring stability of existing judge models is disrupted by scoring biases. Further exploratory experiments and discussions provide valuable insights into the design of scoring prompt templates and the mitigation of scoring biases on aspects such as score rubrics, score IDs, and reference answer selection.

Related papers

• Skewed Score: A statistical framework to assess autograders [2.9645858732618238]
  "LLM-as-a-judge", or autograders, offer a scalable alternative to human evaluation.<n>They have shown mixed reliability and may exhibit systematic biases.<n>We propose a statistical framework that enables researchers to simultaneously assess their autograders.
  arXiv  Detail & Related papers  (2025-07-04T18:45:10Z)
• Pairwise or Pointwise? Evaluating Feedback Protocols for Bias in LLM-Based Evaluation [57.380464382910375]
  We show that the choice of feedback protocol can significantly affect evaluation reliability and induce systematic biases.<n>In particular, we show that pairwise evaluation protocols are more vulnerable to distracted evaluation.
  arXiv  Detail & Related papers  (2025-04-20T19:05:59Z)
• Meta-Evaluating Local LLMs: Rethinking Performance Metrics for Serious Games [3.725822359130832]
  Large Language Models (LLMs) are increasingly being explored as evaluators in serious games.<n>This study investigates the reliability of five small-scale LLMs when assessing player responses in textitEn-join, a game that simulates decision-making within energy communities.<n>Our results highlight the strengths and limitations of each model, revealing trade-offs between sensitivity, specificity, and overall performance.
  arXiv  Detail & Related papers  (2025-04-13T10:46:13Z)
• CompassJudger-1: All-in-one Judge Model Helps Model Evaluation and Evolution [74.41064280094064]
  textbfJudger-1 is the first open-source textbfall-in-one judge LLM. CompassJudger-1 is a general-purpose LLM that demonstrates remarkable versatility. textbfJudgerBench is a new benchmark that encompasses various subjective evaluation tasks.
  arXiv  Detail & Related papers  (2024-10-21T17:56:51Z)
• Diverging Preferences: When do Annotators Disagree and do Models Know? [92.24651142187989]
  We develop a taxonomy of disagreement sources spanning 10 categories across four high-level classes. We find that the majority of disagreements are in opposition with standard reward modeling approaches. We develop methods for identifying diverging preferences to mitigate their influence on evaluation and training.
  arXiv  Detail & Related papers  (2024-10-18T17:32:22Z)
• Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge [84.34545223897578]
  Despite their excellence in many domains, potential issues are under-explored, undermining their reliability and the scope of their utility. We identify 12 key potential biases and propose a new automated bias quantification framework-CALM- which quantifies and analyzes each type of bias in LLM-as-a-Judge. Our work highlights the need for stakeholders to address these issues and remind users to exercise caution in LLM-as-a-Judge applications.
  arXiv  Detail & Related papers  (2024-10-03T17:53:30Z)
• CEB: Compositional Evaluation Benchmark for Fairness in Large Language Models [58.57987316300529]
  Large Language Models (LLMs) are increasingly deployed to handle various natural language processing (NLP) tasks.<n>To evaluate the biases exhibited by LLMs, researchers have recently proposed a variety of datasets.<n>We propose CEB, a Compositional Evaluation Benchmark that covers different types of bias across different social groups and tasks.
  arXiv  Detail & Related papers  (2024-07-02T16:31:37Z)
• The Comparative Trap: Pairwise Comparisons Amplifies Biased Preferences of LLM Evaluators [31.520403357740317]
  Large language models (LLMs) are increasingly used as evaluators for natural language generation tasks.<n>LLMs display biased preferences, such as favoring verbosity and authoritative tones.<n>We introduce PRePair, which integrates pointwise reasoning within a pairwise framework.
  arXiv  Detail & Related papers  (2024-06-18T06:43:04Z)
• Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators [48.54465599914978]
  Large Language Models (LLMs) have demonstrated promising capabilities as automatic evaluators in assessing the quality of generated natural language.<n>LLMs still exhibit biases in evaluation and often struggle to generate coherent evaluations that align with human assessments.<n>We introduce Pairwise-preference Search (PAIRS), an uncertainty-guided search-based rank aggregation method that employs LLMs to conduct pairwise comparisons locally and efficiently ranks candidate texts globally.
  arXiv  Detail & Related papers  (2024-03-25T17:11:28Z)
• Benchmarking Cognitive Biases in Large Language Models as Evaluators [16.845939677403287]
  Large Language Models (LLMs) have been shown to be effective as automatic evaluators with simple prompting and in-context learning. We evaluate the quality of ranking outputs introducing the Cognitive Bias Benchmark for LLMs as Evaluators. We find that LLMs are biased text quality evaluators, exhibiting strong indications on our bias benchmark.
  arXiv  Detail & Related papers  (2023-09-29T06:53:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.