Investigating Gender Bias in LLM-Generated Stories via Psychological Stereotypes

• URL: http://arxiv.org/abs/2508.03292v1
• Date: Tue, 05 Aug 2025 10:10:26 GMT
• Title: Investigating Gender Bias in LLM-Generated Stories via Psychological Stereotypes
• Authors: Shahed Masoudian, Gustavo Escobedo, Hannah Strauss, Markus Schedl,
• Abstract summary: We investigate gender bias in Large Language Models (LLMs) using gender stereotypes studied in psychology.<n>We introduce a novel dataset called StereoBias-Stories containing short stories either unconditioned or conditioned on (one, two, or six) random attributes from 25 psychological stereotypes.<n>We analyze how the gender contribution in the overall story changes in response to these attributes and present three key findings.
• Score: 8.091664636677637
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As Large Language Models (LLMs) are increasingly used across different applications, concerns about their potential to amplify gender biases in various tasks are rising. Prior research has often probed gender bias using explicit gender cues as counterfactual, or studied them in sentence completion and short question answering tasks. These formats might overlook more implicit forms of bias embedded in generative behavior of longer content. In this work, we investigate gender bias in LLMs using gender stereotypes studied in psychology (e.g., aggressiveness or gossiping) in an open-ended task of narrative generation. We introduce a novel dataset called StereoBias-Stories containing short stories either unconditioned or conditioned on (one, two, or six) random attributes from 25 psychological stereotypes and three task-related story endings. We analyze how the gender contribution in the overall story changes in response to these attributes and present three key findings: (1) While models, on average, are highly biased towards male in unconditioned prompts, conditioning on attributes independent from gender stereotypes mitigates this bias. (2) Combining multiple attributes associated with the same gender stereotype intensifies model behavior, with male ones amplifying bias and female ones alleviating it. (3) Model biases align with psychological ground-truth used for categorization, and alignment strength increases with model size. Together, these insights highlight the importance of psychology-grounded evaluation of LLMs.

Related papers

• From Individuals to Interactions: Benchmarking Gender Bias in Multimodal Large Language Models from the Lens of Social Relationship [13.416624729344477]
  We introduce Genres, a novel benchmark designed to evaluate gender bias in MLLMs through the lens of social in relationships generated narratives.<n>Our findings underscore the importance of relationship-aware benchmarks for diagnosing subtle, interaction-driven gender bias in MLLMs.
  arXiv  Detail & Related papers  (2025-06-29T06:03:21Z)
• The LLM Wears Prada: Analysing Gender Bias and Stereotypes through Online Shopping Data [8.26034886618475]
  We investigate whether Large Language Models can predict an individual's gender based solely on online shopping histories.<n>Using a dataset of historical online purchases from users in the United States, we evaluate the ability of six LLMs to classify gender.<n>Results indicate that while models can infer gender with moderate accuracy, their decisions are often rooted in stereotypical associations between product categories and gender.
  arXiv  Detail & Related papers  (2025-04-02T17:56:08Z)
• Revealing and Reducing Gender Biases in Vision and Language Assistants (VLAs) [82.57490175399693]
  We study gender bias in 22 popular image-to-text vision-language assistants (VLAs)<n>Our results show that VLAs replicate human biases likely present in the data, such as real-world occupational imbalances.<n>To eliminate the gender bias in these models, we find that fine-tuning-based debiasing methods achieve the best trade-off between debiasing and retaining performance.
  arXiv  Detail & Related papers  (2024-10-25T05:59:44Z)
• Gender Bias in Decision-Making with Large Language Models: A Study of Relationship Conflicts [15.676219253088211]
  We study gender equity within large language models (LLMs) through a decision-making lens. We explore nine relationship configurations through name pairs across three name lists (men, women, neutral)
  arXiv  Detail & Related papers  (2024-10-14T20:50:11Z)
• Hire Me or Not? Examining Language Model's Behavior with Occupation Attributes [7.718858707298602]
  Large language models (LLMs) have been widely integrated into production pipelines, like recruitment and recommendation systems.<n>This paper investigates LLMs' behavior with respect to gender stereotypes, in the context of occupation decision making.
  arXiv  Detail & Related papers  (2024-05-06T18:09:32Z)
• Disclosure and Mitigation of Gender Bias in LLMs [64.79319733514266]
  Large Language Models (LLMs) can generate biased responses. We propose an indirect probing framework based on conditional generation. We explore three distinct strategies to disclose explicit and implicit gender bias in LLMs.
  arXiv  Detail & Related papers  (2024-02-17T04:48:55Z)
• The Male CEO and the Female Assistant: Evaluation and Mitigation of Gender Biases in Text-To-Image Generation of Dual Subjects [58.27353205269664]
  We propose the Paired Stereotype Test (PST) framework, which queries T2I models to depict two individuals assigned with male-stereotyped and female-stereotyped social identities.<n>PST queries T2I models to depict two individuals assigned with male-stereotyped and female-stereotyped social identities.<n>Using PST, we evaluate two aspects of gender biases -- the well-known bias in gendered occupation and a novel aspect: bias in organizational power.
  arXiv  Detail & Related papers  (2024-02-16T21:32:27Z)
• Probing Explicit and Implicit Gender Bias through LLM Conditional Text Generation [64.79319733514266]
  Large Language Models (LLMs) can generate biased and toxic responses. We propose a conditional text generation mechanism without the need for predefined gender phrases and stereotypes.
  arXiv  Detail & Related papers  (2023-11-01T05:31:46Z)
• Will the Prince Get True Love's Kiss? On the Model Sensitivity to Gender Perturbation over Fairytale Texts [80.21033860436081]
  We investigate how models respond to gender stereotype perturbations through counterfactual data augmentation.<n>Our results show that models exhibit slight performance drops when faced with gender perturbations in the test set.<n>When fine-tuned on counterfactual training data, models become more robust to anti-stereotypical narratives.
  arXiv  Detail & Related papers  (2023-10-16T22:25:09Z)
• Multi-Dimensional Gender Bias Classification [67.65551687580552]
  Machine learning models can inadvertently learn socially undesirable patterns when training on gender biased text. We propose a general framework that decomposes gender bias in text along several pragmatic and semantic dimensions. Using this fine-grained framework, we automatically annotate eight large scale datasets with gender information.
  arXiv  Detail & Related papers  (2020-05-01T21:23:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.