Probing Explicit and Implicit Gender Bias through LLM Conditional Text Generation

• URL: http://arxiv.org/abs/2311.00306v1
• Date: Wed, 1 Nov 2023 05:31:46 GMT
• Title: Probing Explicit and Implicit Gender Bias through LLM Conditional Text Generation
• Authors: Xiangjue Dong, Yibo Wang, Philip S. Yu, James Caverlee
• Abstract summary: Large Language Models (LLMs) can generate biased and toxic responses. We propose a conditional text generation mechanism without the need for predefined gender phrases and stereotypes.
• Score: 64.79319733514266
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Large Language Models (LLMs) can generate biased and toxic responses. Yet most prior work on LLM gender bias evaluation requires predefined gender-related phrases or gender stereotypes, which are challenging to be comprehensively collected and are limited to explicit bias evaluation. In addition, we believe that instances devoid of gender-related language or explicit stereotypes in inputs can still induce gender bias in LLMs. Thus, in this work, we propose a conditional text generation mechanism without the need for predefined gender phrases and stereotypes. This approach employs three types of inputs generated through three distinct strategies to probe LLMs, aiming to show evidence of explicit and implicit gender biases in LLMs. We also utilize explicit and implicit evaluation metrics to evaluate gender bias in LLMs under different strategies. Our experiments demonstrate that an increased model size does not consistently lead to enhanced fairness and all tested LLMs exhibit explicit and/or implicit gender bias, even when explicit gender stereotypes are absent in the inputs.

Related papers

• Gender Bias in LLM-generated Interview Responses [1.6124402884077915]
  This study evaluates three LLMs to conduct a multifaceted audit of LLM-generated interview responses across models, question types, and jobs. Our findings reveal that gender bias is consistent, and closely aligned with gender stereotypes and the dominance of jobs.
  arXiv  Detail & Related papers  (2024-10-28T05:08:08Z)
• GenderCARE: A Comprehensive Framework for Assessing and Reducing Gender Bias in Large Language Models [73.23743278545321]
  Large language models (LLMs) have exhibited remarkable capabilities in natural language generation, but have also been observed to magnify societal biases. GenderCARE is a comprehensive framework that encompasses innovative Criteria, bias Assessment, Reduction techniques, and Evaluation metrics.
  arXiv  Detail & Related papers  (2024-08-22T15:35:46Z)
• From 'Showgirls' to 'Performers': Fine-tuning with Gender-inclusive Language for Bias Reduction in LLMs [1.1049608786515839]
  We adapt linguistic structures within Large Language Models to promote gender-inclusivity. The focus of our work is gender-exclusive affixes in English, such as in'show-girl' or'man-cave'
  arXiv  Detail & Related papers  (2024-07-05T11:31:30Z)
• GenderBias-\emph{VL}: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probing [72.0343083866144]
  This paper introduces the GenderBias-emphVL benchmark to evaluate occupation-related gender bias in Large Vision-Language Models. Using our benchmark, we extensively evaluate 15 commonly used open-source LVLMs and state-of-the-art commercial APIs. Our findings reveal widespread gender biases in existing LVLMs.
  arXiv  Detail & Related papers  (2024-06-30T05:55:15Z)
• Hire Me or Not? Examining Language Model's Behavior with Occupation Attributes [7.718858707298602]
  Large language models (LLMs) have been widely integrated into production pipelines, like recruitment and recommendation systems. This paper investigates LLMs' behavior with respect to gender stereotypes, in the context of occupation decision making.
  arXiv  Detail & Related papers  (2024-05-06T18:09:32Z)
• Disclosure and Mitigation of Gender Bias in LLMs [64.79319733514266]
  Large Language Models (LLMs) can generate biased responses. We propose an indirect probing framework based on conditional generation. We explore three distinct strategies to disclose explicit and implicit gender bias in LLMs.
  arXiv  Detail & Related papers  (2024-02-17T04:48:55Z)
• In-Contextual Gender Bias Suppression for Large Language Models [47.246504807946884]
  Large Language Models (LLMs) have been reported to encode worrying-levels of gender biases. We propose bias suppression that prevents biased generations of LLMs by providing preambles constructed from manually designed templates. We find that bias suppression has acceptable adverse effect on downstream task performance with HellaSwag and COPA.
  arXiv  Detail & Related papers  (2023-09-13T18:39:08Z)
• Multi-Dimensional Gender Bias Classification [67.65551687580552]
  Machine learning models can inadvertently learn socially undesirable patterns when training on gender biased text. We propose a general framework that decomposes gender bias in text along several pragmatic and semantic dimensions. Using this fine-grained framework, we automatically annotate eight large scale datasets with gender information.
  arXiv  Detail & Related papers  (2020-05-01T21:23:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.