Effects of Answer Format Variation on Gender Bias in Large Language Models

• URL: http://arxiv.org/abs/2608.17516v1
• Date: Tue, 18 Aug 2026 08:41:05 GMT
• Title: Effects of Answer Format Variation on Gender Bias in Large Language Models
• Abstract summary: We study how changes in answer format impact the measurement of gender bias in large language models.<n>We find that answer format does substantially alter measured outcomes, including reversals in order rankings.
• Score: 5.457386921498268
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Gender bias or other social biases in large language models (LLMs) are frequently evaluated with question answering or survey benchmarks where the LLM needs to give a response in a predefined answer format. It is well known in survey science that the answer format has a substantial impact on answers, just as LLMs are sensitive to the prompt wording. However, to our knowledge it has not been studied yet how changes in answer format impact the measurement of gender bias in LLMs and their alignment with human response distributions. We evaluate three instruction-tuned models on the BBQ benchmark and OpinionQA survey data across closed-ended, Likert-scaled and open-ended formats, comparing bias measurement and distributional alignment under otherwise identical conditions. We find that answer format does substantially alter measured outcomes, including reversals in order rankings. These differences arise because each format elicits distinct response behaviours, such as forced-choice selection, scale-based distributions and refusal in free-text generation. Our findings highlight the importance of treating answer format as a substantive component of LLM evaluation and motivate multi-format designs for more robust model assessment.

Related papers

• A Consensus-Based Framework for Relative Preference Evaluation of Large Language Models [0.0]
  This paper introduces a consensus-based evaluation framework that measures relative preference among model-generated responses rather than absolute correctness.<n>We conduct a controlled study using five state-of-the-art LLMs across multiple domains, including programming, general knowledge, safety, logical reasoning, and mathematics.<n>Scores are aggregated into a Relative Intelligence Index (RII), representing how frequently a model's responses are preferred by other models.
  arXiv  Detail & Related papers  (2026-07-19T08:59:19Z)
• Prompt Robustness Is Task-Dependent: Comparing Objective and Belief-Style Questions in LLM Evaluation [1.7613484075466104]
  We ask whether prompt robustness differs between objective questions with fixed answers and subjective questions that ask for opinions or values.<n>We evaluate four instruction-tuned model families on three objective datasets and three subjective datasets.<n>Using a binomial generalized estimating equation, we find significant effects of model, dataset, prompt category, and their interactions.
  arXiv  Detail & Related papers  (2026-07-06T18:47:47Z)
• Not What, But How: A Framework for Auditing LLM Responses across Positioning, Generalization, Anthromorphism, and Maxims [54.31707479192195]
  Large language models (LLMs) are being increasingly used to answer subjective, information-seeking questions.<n>Existing evaluations for subjective cultural queries largely focus on factual correctness, ignoring how the response is framed.<n>We introduce FRANZ, an automated FRAmework for respoNse characteriZation to conduct communicative audit of LLM responses along four dimensions.
  arXiv  Detail & Related papers  (2026-06-01T17:01:31Z)
• On the Credibility of Evaluating LLMs using Survey Questions [0.42061757959666934]
  Recent studies evaluate the value orientation of large language models (LLMs) using adapted social surveys.<n>This paper identifies limitations in this methodology that, depending on the exact setup, can lead to both underestimating and overestimating the similarity of value orientation.<n>Using the World Value Survey in three languages across five countries, we demonstrate that prompting methods (direct vs. chain-of-thought) and decoding strategies (greedy vs. sampling) significantly affect results.
  arXiv  Detail & Related papers  (2026-02-03T21:45:43Z)
• Prompt Perturbations Reveal Human-Like Biases in Large Language Model Survey Responses [2.3112192919085826]
  Large Language Models (LLMs) are increasingly used as proxies for human subjects in social science surveys.<n>Their reliability and susceptibility to known human-like response biases are poorly understood.<n>This work investigates the response robustness of LLMs in normative survey contexts.
  arXiv  Detail & Related papers  (2025-07-09T18:01:50Z)
• Do LLMs have a Gender (Entropy) Bias? [3.2225437367979763]
  We define and study entropy bias, which we define as a discrepancy in the amount of information generated by an LLM in response to real questions users have asked.<n>Our analyses suggest that there is no significant bias in LLM responses for men and women at a category level.<n>We suggest a simple debiasing approach that iteratively merges the responses for the two genders to produce a final result.
  arXiv  Detail & Related papers  (2025-05-24T23:06:41Z)
• Right Answer, Wrong Score: Uncovering the Inconsistencies of LLM Evaluation in Multiple-Choice Question Answering [78.89231943329885]
  Multiple-Choice Question Answering (MCQA) is widely used to evaluate Large Language Models (LLMs)<n>We show that multiple factors can significantly impact the reported performance of LLMs.<n>We analyze whether existing answer extraction methods are aligned with human judgment.
  arXiv  Detail & Related papers  (2025-03-19T08:45:03Z)
• LLMs Are Biased Towards Output Formats! Systematically Evaluating and Mitigating Output Format Bias of LLMs [69.40865293066885]
  We present the first systematic evaluation examining format bias in performance of large language models (LLMs)<n>We present our empirical format bias evaluation spanning four commonly used categories -- multiple-choice question-answer, wrapping, list, and mapping.
  arXiv  Detail & Related papers  (2024-08-16T10:45:45Z)
• From Distributional to Overton Pluralism: Investigating Large Language Model Alignment [82.99849359892112]
  We re-examine previously reported reductions in response diversity post-alignment.<n>Our analysis suggests that an apparent drop in the diversity of responses is largely explained by quality control and information aggregation.<n>Findings indicate that current alignment techniques capture but do not extend the useful subset of assistant-like base LLM behavior.
  arXiv  Detail & Related papers  (2024-06-25T16:32:33Z)
• Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models [61.45529177682614]
  We challenge the prevailing constrained evaluation paradigm for values and opinions in large language models. We show that models give substantively different answers when not forced. We distill these findings into recommendations and open challenges in evaluating values and opinions in LLMs.
  arXiv  Detail & Related papers  (2024-02-26T18:00:49Z)
• Do LLMs exhibit human-like response biases? A case study in survey design [66.1850490474361]
  We investigate the extent to which large language models (LLMs) reflect human response biases, if at all. We design a dataset and framework to evaluate whether LLMs exhibit human-like response biases in survey questionnaires. Our comprehensive evaluation of nine models shows that popular open and commercial LLMs generally fail to reflect human-like behavior.
  arXiv  Detail & Related papers  (2023-11-07T15:40:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.