With a Grain of SALT: Are LLMs Fair Across Social Dimensions?

• URL: http://arxiv.org/abs/2410.12499v1
• Date: Wed, 16 Oct 2024 12:22:47 GMT
• Title: With a Grain of SALT: Are LLMs Fair Across Social Dimensions?
• Authors: Samee Arif, Zohaib Khan, Agha Ali Raza, Awais Athar,
• Abstract summary: This paper presents an analysis of biases in open-source Large Language Models (LLMs) across various genders, religions, and races. We introduce a methodology for generating a bias detection dataset using seven bias triggers: General Debate, Positioned Debate, Career Advice, Story Generation, Problem-Solving, Cover-Letter Writing, and CV Generation. We anonymise the LLM-generated text associated with each group using GPT-4o-mini and do a pairwise comparison using GPT-4o-as-a-Judge.
• Score: 3.979019316355144
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents an analysis of biases in open-source Large Language Models (LLMs) across various genders, religions, and races. We introduce a methodology for generating a bias detection dataset using seven bias triggers: General Debate, Positioned Debate, Career Advice, Story Generation, Problem-Solving, Cover-Letter Writing, and CV Generation. We use GPT-4o to generate a diverse set of prompts for each trigger across various genders, religious and racial groups. We evaluate models from Llama and Gemma family on the generated dataset. We anonymise the LLM-generated text associated with each group using GPT-4o-mini and do a pairwise comparison using GPT-4o-as-a-Judge. To quantify bias in the LLM-generated text we use the number of wins and losses in the pairwise comparison. Our analysis spans three languages, English, German, and Arabic to explore how language influences bias manifestation. Our findings reveal that LLMs exhibit strong polarization toward certain groups across each category, with a notable consistency observed across models. However, when switching languages, variations and anomalies emerge, often attributable to cultural cues and contextual differences.

Related papers

• Rethinking Prompt-based Debiasing in Large Language Models [40.90578215191079]
  Investigating bias in large language models (LLMs) is crucial for developing trustworthy AI. While prompt-based through prompt engineering is common, its effectiveness relies on the assumption that models inherently understand biases. Our study systematically analyzed this assumption using the BBQ and StereoSet benchmarks on both open-source models as well as commercial GPT model.
  arXiv  Detail & Related papers  (2025-03-12T10:06:03Z)
• Towards Resource Efficient and Interpretable Bias Mitigation in Large Language Models [1.787433808079955]
  Large language models (LLMs) have been observed to perpetuate unwanted biases in training data. In this paper, we mitigate bias by leveraging small biased and anti-biased expert models to obtain a debiasing signal. Experiments on mitigating gender, race, and religion biases show a reduction in bias on several local and global bias metrics.
  arXiv  Detail & Related papers  (2024-12-02T16:56:08Z)
• Attention Speaks Volumes: Localizing and Mitigating Bias in Language Models [15.53216696218776]
  We explore the internal mechanisms of how bias emerges in large language models (LLMs) when provided with ambiguous comparative prompts. We propose $textttATLAS$, a technique to localize bias to specific layers of the LLM by analyzing attention scores and then reduce bias by scaling attention in these biased layers.
  arXiv  Detail & Related papers  (2024-10-29T20:15:56Z)
• Large Language Models Reflect the Ideology of their Creators [73.25935570218375]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language. We uncover notable diversity in the ideological stance exhibited across different LLMs and languages.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• Identifying and Mitigating Social Bias Knowledge in Language Models [52.52955281662332]
  We propose a novel debiasing approach, Fairness Stamp (FAST), which enables fine-grained calibration of individual social biases. FAST surpasses state-of-the-art baselines with superior debiasing performance. This highlights the potential of fine-grained debiasing strategies to achieve fairness in large language models.
  arXiv  Detail & Related papers  (2024-08-07T17:14:58Z)
• GenderBias-\emph{VL}: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probing [72.0343083866144]
  This paper introduces the GenderBias-emphVL benchmark to evaluate occupation-related gender bias in Large Vision-Language Models. Using our benchmark, we extensively evaluate 15 commonly used open-source LVLMs and state-of-the-art commercial APIs. Our findings reveal widespread gender biases in existing LVLMs.
  arXiv  Detail & Related papers  (2024-06-30T05:55:15Z)
• Do they mean 'us'? Interpreting Referring Expressions in Intergroup Bias [42.35739515777376]
  In this paper, we model the intergroup bias as a tagging task on English sports comments from forums dedicated to fandom for NFL teams. We curate a unique dataset of over 6 million game-time comments from opposing perspectives (the teams in the game) Expert and crowd annotations justify modeling the bias through tagging of implicit and explicit referring expressions.
  arXiv  Detail & Related papers  (2024-06-25T21:47:53Z)
• VLBiasBench: A Comprehensive Benchmark for Evaluating Bias in Large Vision-Language Model [72.13121434085116]
  We introduce VLBiasBench, a benchmark to evaluate biases in Large Vision-Language Models (LVLMs) VLBiasBench features a dataset that covers nine distinct categories of social biases, including age, disability status, gender, nationality, physical appearance, race, religion, profession, social economic status, as well as two intersectional bias categories: race x gender and race x social economic status. We conduct extensive evaluations on 15 open-source models as well as two advanced closed-source models, yielding new insights into the biases present in these models.
  arXiv  Detail & Related papers  (2024-06-20T10:56:59Z)
• JobFair: A Framework for Benchmarking Gender Hiring Bias in Large Language Models [12.12628747941818]
  This paper presents a novel framework for benchmarking hierarchical gender hiring bias in Large Language Models (LLMs) for resume scoring. We introduce a new construct grounded in labour economics, legal principles, and critiques of current bias benchmarks. We analyze gender hiring biases in ten state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2024-06-17T09:15:57Z)
• White Men Lead, Black Women Help? Benchmarking Language Agency Social Biases in LLMs [58.27353205269664]
  Social biases can manifest in language agency. We introduce the novel Language Agency Bias Evaluation benchmark. We unveil language agency social biases in 3 recent Large Language Model (LLM)-generated content.
  arXiv  Detail & Related papers  (2024-04-16T12:27:54Z)
• Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinement [75.7148545929689]
  Large language models (LLMs) improve their performance through self-feedback on certain tasks while degrade on others. We formally define LLM's self-bias - the tendency to favor its own generation. We analyze six LLMs on translation, constrained text generation, and mathematical reasoning tasks.
  arXiv  Detail & Related papers  (2024-02-18T03:10:39Z)
• GPTBIAS: A Comprehensive Framework for Evaluating Bias in Large Language Models [83.30078426829627]
  Large language models (LLMs) have gained popularity and are being widely adopted by a large user community. The existing evaluation methods have many constraints, and their results exhibit a limited degree of interpretability. We propose a bias evaluation framework named GPTBIAS that leverages the high performance of LLMs to assess bias in models.
  arXiv  Detail & Related papers  (2023-12-11T12:02:14Z)
• Aligning with Whom? Large Language Models Have Gender and Racial Biases in Subjective NLP Tasks [15.015148115215315]
  We conduct experiments on four popular large language models (LLMs) to investigate their capability to understand group differences and potential biases in their predictions for politeness and offensiveness. We find that for both tasks, model predictions are closer to the labels from White and female participants. More specifically, when being prompted to respond from the perspective of "Black" and "Asian" individuals, models show lower performance in predicting both overall scores as well as the scores from corresponding groups.
  arXiv  Detail & Related papers  (2023-11-16T10:02:24Z)
• Fair Abstractive Summarization of Diverse Perspectives [103.08300574459783]
  A fair summary should provide a comprehensive coverage of diverse perspectives without underrepresenting certain groups. We first formally define fairness in abstractive summarization as not underrepresenting perspectives of any groups of people. We propose four reference-free automatic metrics by measuring the differences between target and source perspectives.
  arXiv  Detail & Related papers  (2023-11-14T03:38:55Z)
• Do Not Harm Protected Groups in Debiasing Language Representation Models [2.9057513016551244]
  Language Representation Models (LRMs) trained with real-world data may capture and exacerbate undesired bias. We examine four debiasing techniques on a real-world text classification task and show that reducing biasing is at the cost of degrading performance for all demographic groups.
  arXiv  Detail & Related papers  (2023-10-27T20:11:38Z)
• CALM : A Multi-task Benchmark for Comprehensive Assessment of Language Model Bias [7.28980829208179]
  Comprehensive Assessment of Language Models (CALM) for robust measurement of two types of universally relevant sociodemographic bias, gender and race. Our empirical evaluation shows that CALM bias scores are more robust and far less sensitive than previous bias measurements to perturbations in the templates.
  arXiv  Detail & Related papers  (2023-08-24T03:53:55Z)
• Queer People are People First: Deconstructing Sexual Identity Stereotypes in Large Language Models [3.974379576408554]
  Large Language Models (LLMs) are trained primarily on minimally processed web text. LLMs can inadvertently perpetuate stereotypes towards marginalized groups, like the LGBTQIA+ community.
  arXiv  Detail & Related papers  (2023-06-30T19:39:01Z)
• Comparing Biases and the Impact of Multilingual Training across Multiple Languages [70.84047257764405]
  We present a bias analysis across Italian, Chinese, English, Hebrew, and Spanish on the downstream sentiment analysis task. We adapt existing sentiment bias templates in English to Italian, Chinese, Hebrew, and Spanish for four attributes: race, religion, nationality, and gender. Our results reveal similarities in bias expression such as favoritism of groups that are dominant in each language's culture.
  arXiv  Detail & Related papers  (2023-05-18T18:15:07Z)
• Challenges in Measuring Bias via Open-Ended Language Generation [1.5552869983952944]
  We analyze how specific choices of prompt sets, metrics, automatic tools and sampling strategies affect bias results. We provide recommendations for reporting biases in open-ended language generation.
  arXiv  Detail & Related papers  (2022-05-23T19:57:15Z)
• Balancing out Bias: Achieving Fairness Through Training Reweighting [58.201275105195485]
  Bias in natural language processing arises from models learning characteristics of the author such as gender and race. Existing methods for mitigating and measuring bias do not directly account for correlations between author demographics and linguistic variables. This paper introduces a very simple but highly effective method for countering bias using instance reweighting.
  arXiv  Detail & Related papers  (2021-09-16T23:40:28Z)
• Towards Controllable Biases in Language Generation [87.89632038677912]
  We develop a method to induce societal biases in generated text when input prompts contain mentions of specific demographic groups. We analyze two scenarios: 1) inducing negative biases for one demographic and positive biases for another demographic, and 2) equalizing biases between demographics.
  arXiv  Detail & Related papers  (2020-05-01T08:25:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.