Developing A Framework to Support Human Evaluation of Bias in Generated Free Response Text

• URL: http://arxiv.org/abs/2505.03053v1
• Date: Mon, 05 May 2025 22:26:55 GMT
• Title: Developing A Framework to Support Human Evaluation of Bias in Generated Free Response Text
• Authors: Jennifer Healey, Laurie Byrum, Md Nadeem Akhtar, Surabhi Bhargava, Moumita Sinha,
• Abstract summary: We present our journey towards developing a semi-automated bias evaluation framework for free text responses.<n>We discuss how we developed an operational definition of bias that helped us automate our pipeline and a methodology for classifying bias beyond multiple choice.
• Score: 8.41305848182636
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: LLM evaluation is challenging even the case of base models. In real world deployments, evaluation is further complicated by the interplay of task specific prompts and experiential context. At scale, bias evaluation is often based on short context, fixed choice benchmarks that can be rapidly evaluated, however, these can lose validity when the LLMs' deployed context differs. Large scale human evaluation is often seen as too intractable and costly. Here we present our journey towards developing a semi-automated bias evaluation framework for free text responses that has human insights at its core. We discuss how we developed an operational definition of bias that helped us automate our pipeline and a methodology for classifying bias beyond multiple choice. We additionally comment on how human evaluation helped us uncover problematic templates in a bias benchmark.

Related papers

• Real-World Summarization: When Evaluation Reaches Its Limits [1.4197924572122094]
  We compare traditional metrics, trainable methods, and LLM-as-a-judge approaches.<n>Our findings reveal that simpler metrics like word overlap surprisingly well with human judgments.<n>Our analysis of real-world business impacts shows incorrect and non-checkable information pose the greatest risks.
  arXiv  Detail & Related papers  (2025-07-15T17:23:56Z)
• Monocle: Hybrid Local-Global In-Context Evaluation for Long-Text Generation with Uncertainty-Based Active Learning [63.531262595858]
  Divide-and-conquer approach breaks comprehensive evaluation task into localized scoring tasks, followed by a final global assessment.<n>We introduce a hybrid in-context learning approach that leverages human annotations to enhance the performance of both local and global evaluations.<n>Finally, we develop an uncertainty-based active learning algorithm that efficiently selects data samples for human annotation.
  arXiv  Detail & Related papers  (2025-05-26T16:39:41Z)
• Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settings [36.449658676568234]
  Large language model (LLM)-as-judge paradigm has been used to meet the demand for a cheap, reliable, and fast evaluation of model outputs.<n>We propose ContextualJudgeBench, a judge benchmark with 2,000 challenging response pairs across eight splits inspired by real-world contextual evaluation scenarios.<n>Our comprehensive study reveals that the contextual information and its assessment criteria present a significant challenge to even state-of-the-art models.
  arXiv  Detail & Related papers  (2025-03-19T18:09:19Z)
• EQUATOR: A Deterministic Framework for Evaluating LLM Reasoning with Open-Ended Questions. # v1.0.0-beta [2.1249213103048414]
  We introduce the EQUATOR Evaluator, which combines deterministic scoring with a focus on factual accuracy and robust reasoning assessment.<n>Using a vector database, EQUATOR pairs open-ended questions with human-evaluated answers, enabling more precise and scalable evaluations.<n>Our results demonstrate that this framework significantly outperforms traditional multiple-choice evaluations while maintaining high accuracy standards.
  arXiv  Detail & Related papers  (2024-12-31T03:56:17Z)
• HREF: Human Response-Guided Evaluation of Instruction Following in Language Models [61.273153125847166]
  We develop a new evaluation benchmark, Human Response-Guided Evaluation of Instruction Following (HREF)<n>In addition to providing reliable evaluation, HREF emphasizes individual task performance and is free from contamination.<n>We study the impact of key design choices in HREF, including the size of the evaluation set, the judge model, the baseline model, and the prompt template.
  arXiv  Detail & Related papers  (2024-12-20T03:26:47Z)
• Towards Understanding the Robustness of LLM-based Evaluations under Perturbations [9.944512689015998]
  Large Language Models (LLMs) can serve as automatic evaluators for non-standardized metrics in summarization and dialog-based tasks.<n>We conduct experiments across multiple prompting strategies to examine how LLMs fare as quality evaluators when compared with human judgments.
  arXiv  Detail & Related papers  (2024-12-12T13:31:58Z)
• A Framework for Evaluating LLMs Under Task Indeterminacy [49.298107503257036]
  Large language model (LLM) evaluations often assume there is a single correct response -- a gold label -- for each item in the evaluation corpus. We develop a framework for evaluating LLMs under task indeterminacy.
  arXiv  Detail & Related papers  (2024-11-21T00:15:44Z)
• CompassJudger-1: All-in-one Judge Model Helps Model Evaluation and Evolution [74.41064280094064]
  textbfJudger-1 is the first open-source textbfall-in-one judge LLM. CompassJudger-1 is a general-purpose LLM that demonstrates remarkable versatility. textbfJudgerBench is a new benchmark that encompasses various subjective evaluation tasks.
  arXiv  Detail & Related papers  (2024-10-21T17:56:51Z)
• With a Grain of SALT: Are LLMs Fair Across Social Dimensions? [3.5001789247699535]
  This paper presents a systematic analysis of biases in open-source Large Language Models (LLMs) across gender, religion, and race.<n>We use the SALT dataset, which incorporates five distinct bias triggers: General Debate, Positioned Debate, Career Advice, Problem Solving, and CV Generation.<n>Our findings reveal consistent polarization across models, with certain demographic groups receiving systematically favorable or unfavorable treatment.
  arXiv  Detail & Related papers  (2024-10-16T12:22:47Z)
• Reference-Guided Verdict: LLMs-as-Judges in Automatic Evaluation of Free-Form Text [12.879551933541345]
  Large Language Models (LLMs) are capable of generating human-like conversations. Conventional metrics like BLEU and ROUGE are inadequate for capturing the subtle semantics and contextual richness of such generative outputs. We propose a reference-guided verdict method that automates the evaluation process by leveraging multiple LLMs-as-judges.
  arXiv  Detail & Related papers  (2024-08-17T16:01:45Z)
• Bias in Language Models: Beyond Trick Tests and Toward RUTEd Evaluation [49.3814117521631]
  Standard benchmarks of bias and fairness in large language models (LLMs) measure the association between social attributes implied in user prompts and short responses.<n>We develop analogous RUTEd evaluations from three contexts of real-world use.<n>We find that standard bias metrics have no significant correlation with the more realistic bias metrics.
  arXiv  Detail & Related papers  (2024-02-20T01:49:15Z)
• GPTBIAS: A Comprehensive Framework for Evaluating Bias in Large Language Models [83.30078426829627]
  Large language models (LLMs) have gained popularity and are being widely adopted by a large user community. The existing evaluation methods have many constraints, and their results exhibit a limited degree of interpretability. We propose a bias evaluation framework named GPTBIAS that leverages the high performance of LLMs to assess bias in models.
  arXiv  Detail & Related papers  (2023-12-11T12:02:14Z)
• Style Over Substance: Evaluation Biases for Large Language Models [17.13064447978519]
  This study investigates the behavior of crowd-sourced and expert annotators, as well as large language models (LLMs) Our findings reveal a concerning bias in the evaluation process, as answers with factual errors are rated more favorably than answers that are too short or contained grammatical errors. We propose independently evaluating machine-generated text across multiple dimensions, rather than merging all the evaluation aspects into a single score.
  arXiv  Detail & Related papers  (2023-07-06T14:42:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.