Prompt and Prejudice

• URL: http://arxiv.org/abs/2408.04671v1
• Date: Wed, 7 Aug 2024 14:11:33 GMT
• Title: Prompt and Prejudice
• Authors: Lorenzo Berlincioni, Luca Cultrera, Federico Becattini, Marco Bertini, Alberto Del Bimbo,
• Abstract summary: This paper investigates the impact of using first names in Large Language Models (LLMs) and Vision Language Models (VLMs) We propose an approach that appends first names to ethically annotated text scenarios to reveal demographic biases in model outputs.
• Score: 29.35618753825668
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper investigates the impact of using first names in Large Language Models (LLMs) and Vision Language Models (VLMs), particularly when prompted with ethical decision-making tasks. We propose an approach that appends first names to ethically annotated text scenarios to reveal demographic biases in model outputs. Our study involves a curated list of more than 300 names representing diverse genders and ethnic backgrounds, tested across thousands of moral scenarios. Following the auditing methodologies from social sciences we propose a detailed analysis involving popular LLMs/VLMs to contribute to the field of responsible AI by emphasizing the importance of recognizing and mitigating biases in these systems. Furthermore, we introduce a novel benchmark, the Pratical Scenarios Benchmark (PSB), designed to assess the presence of biases involving gender or demographic prejudices in everyday decision-making scenarios as well as practical scenarios where an LLM might be used to make sensible decisions (e.g., granting mortgages or insurances). This benchmark allows for a comprehensive comparison of model behaviors across different demographic categories, highlighting the risks and biases that may arise in practical applications of LLMs and VLMs.

Related papers

• An Actionable Framework for Assessing Bias and Fairness in Large Language Model Use Cases [0.0]
  This paper aims to provide a technical guide for practitioners to assess bias and fairness risks in large language models. The main contribution of this work is a decision framework that allows practitioners to determine which metrics to use for a specific LLM use case.
  arXiv  Detail & Related papers  (2024-07-15T16:04:44Z)
• The African Woman is Rhythmic and Soulful: An Investigation of Implicit Biases in LLM Open-ended Text Generation [3.9945212716333063]
  Implicit biases are significant because they influence the decisions made by Large Language Models (LLMs) Traditionally, explicit bias tests or embedding-based methods are employed to detect bias, but these approaches can overlook more nuanced, implicit forms of bias. We introduce two novel psychological-inspired methodologies to reveal and measure implicit biases through prompt-based and decision-making tasks.
  arXiv  Detail & Related papers  (2024-07-01T13:21:33Z)
• GenderBias-\emph{VL}: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probing [72.0343083866144]
  This paper introduces the GenderBias-emphVL benchmark to evaluate occupation-related gender bias in Large Vision-Language Models. Using our benchmark, we extensively evaluate 15 commonly used open-source LVLMs and state-of-the-art commercial APIs. Our findings reveal widespread gender biases in existing LVLMs.
  arXiv  Detail & Related papers  (2024-06-30T05:55:15Z)
• Evaluating Human Alignment and Model Faithfulness of LLM Rationale [66.75309523854476]
  We study how well large language models (LLMs) explain their generations through rationales. We show that prompting-based methods are less "faithful" than attribution-based explanations.
  arXiv  Detail & Related papers  (2024-06-28T20:06:30Z)
• Categorical Syllogisms Revisited: A Review of the Logical Reasoning Abilities of LLMs for Analyzing Categorical Syllogism [62.571419297164645]
  This paper provides a systematic overview of prior works on the logical reasoning ability of large language models for analyzing categorical syllogisms. We first investigate all the possible variations for the categorical syllogisms from a purely logical perspective. We then examine the underlying configurations (i.e., mood and figure) tested by the existing datasets.
  arXiv  Detail & Related papers  (2024-06-26T21:17:20Z)
• Understanding Intrinsic Socioeconomic Biases in Large Language Models [4.276697874428501]
  We introduce a novel dataset of one million English sentences to quantify socioeconomic biases. Our findings reveal pervasive socioeconomic biases in both established models like GPT-2 and state-of-the-art models like Llama 2 and Falcon.
  arXiv  Detail & Related papers  (2024-05-28T23:54:44Z)
• Prejudice and Volatility: A Statistical Framework for Measuring Social Discrimination in Large Language Models [0.0]
  This study investigates why and how inconsistency in the generation of Large Language Models (LLMs) might induce or exacerbate societal injustice. We formulate the Prejudice-Volatility Framework (PVF) that precisely defines behavioral metrics for assessing LLMs. We mathematically dissect the aggregated discrimination risk of LLMs into prejudice risk, originating from their system bias, and volatility risk.
  arXiv  Detail & Related papers  (2024-02-23T18:15:56Z)
• Exploring Value Biases: How LLMs Deviate Towards the Ideal [57.99044181599786]
  Large-Language-Models (LLMs) are deployed in a wide range of applications, and their response has an increasing social impact. We show that value bias is strong in LLMs across different categories, similar to the results found in human studies.
  arXiv  Detail & Related papers  (2024-02-16T18:28:43Z)
• Survey of Social Bias in Vision-Language Models [65.44579542312489]
  Survey aims to provide researchers with a high-level insight into the similarities and differences of social bias studies in pre-trained models across NLP, CV, and VL. The findings and recommendations presented here can benefit the ML community, fostering the development of fairer and non-biased AI models.
  arXiv  Detail & Related papers  (2023-09-24T15:34:56Z)
• Bias and Fairness in Large Language Models: A Survey [73.87651986156006]
  We present a comprehensive survey of bias evaluation and mitigation techniques for large language models (LLMs) We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing. We then unify the literature by proposing three intuitive, two for bias evaluation, and one for mitigation.
  arXiv  Detail & Related papers  (2023-09-02T00:32:55Z)
• The Unequal Opportunities of Large Language Models: Revealing Demographic Bias through Job Recommendations [5.898806397015801]
  We propose a simple method for analyzing and comparing demographic bias in Large Language Models (LLMs) We demonstrate the effectiveness of our method by measuring intersectional biases within ChatGPT and LLaMA. We identify distinct biases in both models toward various demographic identities, such as both models consistently suggesting low-paying jobs for Mexican workers.
  arXiv  Detail & Related papers  (2023-08-03T21:12:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.