Humans or LLMs as the Judge? A Study on Judgement Biases

• URL: http://arxiv.org/abs/2402.10669v5
• Date: Thu, 26 Sep 2024 03:16:52 GMT
• Title: Humans or LLMs as the Judge? A Study on Judgement Biases
• Authors: Guiming Hardy Chen, Shunian Chen, Ziche Liu, Feng Jiang, Benyou Wang,
• Abstract summary: We propose a novel framework that is free from referencing groundtruth annotations for investigating Misinformation Oversight Bias, Gender Bias, Authority Bias and Beauty Bias on LLM and human judges. Results show that human and LLM judges are vulnerable to perturbations to various degrees, and that even the cutting-edge judges possess considerable biases.
• Score: 17.069314000437537
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Adopting human and large language models (LLM) as judges (a.k.a human- and LLM-as-a-judge) for evaluating the performance of LLMs has recently gained attention. Nonetheless, this approach concurrently introduces potential biases from human and LLMs, questioning the reliability of the evaluation results. In this paper, we propose a novel framework that is free from referencing groundtruth annotations for investigating Misinformation Oversight Bias, Gender Bias, Authority Bias and Beauty Bias on LLM and human judges. We curate a dataset referring to the revised Bloom's Taxonomy and conduct thousands of evaluations. Results show that human and LLM judges are vulnerable to perturbations to various degrees, and that even the cutting-edge judges possess considerable biases. We further exploit these biases to conduct attacks on LLM judges. We hope that our work can notify the community of the bias and vulnerability of human- and LLM-as-a-judge, as well as the urgency of developing robust evaluation systems.

Related papers

• From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge [32.55871325700294]
  Assessment and evaluation have long been critical challenges in artificial intelligence (AI) and natural language processing (NLP) Recent advancements in Large Language Models (LLMs) inspire the "LLM-as-a-judge" paradigm.
  arXiv  Detail & Related papers  (2024-11-25T17:28:44Z)
• JudgeBench: A Benchmark for Evaluating LLM-based Judges [61.048125269475854]
  JudgeBench is a benchmark for evaluating LLM-based judges on challenging response pairs spanning knowledge, reasoning, math, and coding. Our comprehensive evaluation on a collection of prompted judges, fine-tuned judges, multi-agent judges, and reward models shows that JudgeBench poses a significantly greater challenge than previous benchmarks.
  arXiv  Detail & Related papers  (2024-10-16T17:58:19Z)
• Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge [84.34545223897578]
  Despite their excellence in many domains, potential issues are under-explored, undermining their reliability and the scope of their utility. We identify 12 key potential biases and propose a new automated bias quantification framework-CALM- which quantifies and analyzes each type of bias in LLM-as-a-Judge. Our work highlights the need for stakeholders to address these issues and remind users to exercise caution in LLM-as-a-Judge applications.
  arXiv  Detail & Related papers  (2024-10-03T17:53:30Z)
• Systematic Evaluation of LLM-as-a-Judge in LLM Alignment Tasks: Explainable Metrics and Diverse Prompt Templates [10.091146498861333]
  Commercial large language models (LLMs) like GPT-4 have been recently employed to evaluate and compare different alignment approaches. We develop a framework to evaluate, compare, and visualize the reliability and alignment of LLM judges.
  arXiv  Detail & Related papers  (2024-08-23T11:49:01Z)
• Evaluating the Evaluator: Measuring LLMs' Adherence to Task Evaluation Instructions [18.93335792080899]
  We investigate how much influence prompting the LLMs-as-a-judge has on the alignment of AI judgements to human judgements. We aggregate a taxonomy of quality criteria commonly used across state-of-the-art evaluations with LLMs and provide this as a rigorous benchmark of models as judges.
  arXiv  Detail & Related papers  (2024-08-16T14:49:35Z)
• Decoding Biases: Automated Methods and LLM Judges for Gender Bias Detection in Language Models [47.545382591646565]
  Large Language Models (LLMs) have excelled at language understanding and generating human-level text. LLMs are susceptible to adversarial attacks where malicious users prompt the model to generate undesirable text. In this work, we train models to automatically create adversarial prompts to elicit biased responses from target LLMs.
  arXiv  Detail & Related papers  (2024-08-07T17:11:34Z)
• LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks [106.09361690937618]
  There is an increasing trend towards evaluating NLP models with LLM-generated judgments instead of human judgments. In the absence of a comparison against human data, this raises concerns about the validity of these evaluations. We provide JUDGE-BENCH, a collection of 20 NLP datasets with human annotations.
  arXiv  Detail & Related papers  (2024-06-26T14:56:13Z)
• JudgeLM: Fine-tuned Large Language Models are Scalable Judges [54.007823006976516]
  We propose to fine-tune Large Language Models (LLMs) as scalable judges (JudgeLM) to evaluate LLMs efficiently and effectively in open-ended benchmarks. We first propose a comprehensive, large-scale, high-quality dataset containing task seeds, LLMs-generated answers, and GPT-4-generated judgments for fine-tuning high-performance judges. We then analyze the key biases in fine-tuning LLM as a judge and consider them as position bias, knowledge bias, and format bias.
  arXiv  Detail & Related papers  (2023-10-26T17:48:58Z)
• Can Large Language Models Be an Alternative to Human Evaluations? [80.81532239566992]
  Large language models (LLMs) have demonstrated exceptional performance on unseen tasks when only the task instructions are provided. We show that the result of LLM evaluation is consistent with the results obtained by expert human evaluation.
  arXiv  Detail & Related papers  (2023-05-03T07:28:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.