Contaminated Collaboration: Measuring Gender Bias Transfer in LLM-Assisted Student Writing

• URL: http://arxiv.org/abs/2606.15914v1
• Date: Sun, 14 Jun 2026 16:44:36 GMT
• Title: Contaminated Collaboration: Measuring Gender Bias Transfer in LLM-Assisted Student Writing
• Abstract summary: We investigate whether gender bias in an LLM writing assistant transfers into career plan essays written by students.<n>Students in the biased condition produced essays with a significantly larger agentic gap and more gender-stereotypic occupation suggestions than those in the control and neutral conditions.<n>Our results also reveal that this bias transfer is asymmetric: agency is suppressed in female-target essays while male-target writing remains largely unaffected.
• Score: 1.8393763289985177
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Gender bias in LLMs has been studied extensively in model outputs, with biased prompts shown to amplify stereotyped generations. Whether such bias propagates into text produced by humans who use these systems, however, remains underexplored. We investigate whether gender bias in an LLM writing assistant transfers into career plan essays written by students. We first verify that a gender-biased prompt induces gender-differentiated language in LLM-generated essays, while a neutral prompt does not. We then recruited participants (N = 123) in a controlled environment to write career plan essays for paired biographical profiles differing only in gender under three conditions: no AI assistance, neutral LLM assistance, or gender-biased LLM assistance. Students in the biased condition produced essays with a significantly larger agentic gap and more gender-stereotypic occupation suggestions than those in the control and neutral conditions. Our results also reveal that this bias transfer is asymmetric: agency is suppressed in female-target essays while male-target writing remains largely unaffected. Our findings highlight the risk of bias propagation in AI-assisted writing, calling for fairness-aware design in educational AI tools.

Related papers

• BioPro: On Difference-Aware Gender Fairness for Vision-Language Models [50.40913324046528]
  Vision-Language Models (VLMs) inherit significant social biases from their training data, notably in gender representation.<n>We build upon recent advances in difference-aware fairness for text-only models to formalize the problem of difference-aware gender fairness for image captioning and text-to-image generation.<n>We propose BioPro, which aims to mitigate unwanted bias in neutral contexts while preserving valid distinctions in explicit ones.
  arXiv  Detail & Related papers  (2025-11-30T09:33:09Z)
• More Women, Same Stereotypes: Unpacking the Gender Bias Paradox in Large Language Models [3.154053412440065]
  Large Language Models (LLMs) have revolutionized natural language processing, yet concerns persist regarding their tendency to reflect or amplify social biases.<n>This study introduces a novel evaluation framework to uncover gender biases in LLMs.<n>A systematic analysis of ten prominent LLMs shows a consistent pattern of overrepresenting female characters across occupations.
  arXiv  Detail & Related papers  (2025-03-20T07:15:45Z)
• Gender Bias in LLM-generated Interview Responses [1.6124402884077915]
  This study evaluates three LLMs to conduct a multifaceted audit of LLM-generated interview responses across models, question types, and jobs.<n>Our findings reveal that gender bias is consistent, and closely aligned with gender stereotypes and the dominance of jobs.
  arXiv  Detail & Related papers  (2024-10-28T05:08:08Z)
• White Men Lead, Black Women Help? Benchmarking and Mitigating Language Agency Social Biases in LLMs [58.27353205269664]
  Social biases can manifest in language agency in Large Language Model (LLM)-generated content.<n>We introduce the Language Agency Bias Evaluation benchmark, which comprehensively evaluates biases in LLMs.<n>Using LABE, we unveil language agency social biases in 3 recent LLMs: ChatGPT, Llama3, and Mistral.
  arXiv  Detail & Related papers  (2024-04-16T12:27:54Z)
• Disclosure and Mitigation of Gender Bias in LLMs [64.79319733514266]
  Large Language Models (LLMs) can generate biased responses. We propose an indirect probing framework based on conditional generation. We explore three distinct strategies to disclose explicit and implicit gender bias in LLMs.
  arXiv  Detail & Related papers  (2024-02-17T04:48:55Z)
• Unraveling Downstream Gender Bias from Large Language Models: A Study on AI Educational Writing Assistance [13.276943737418417]
  Large Language Models (LLMs) are increasingly utilized in educational tasks such as providing writing suggestions to students. Previous studies have investigated bias in models and data representations separately. We investigate how bias transfers through an AI writing support pipeline.
  arXiv  Detail & Related papers  (2023-11-06T18:01:34Z)
• Probing Explicit and Implicit Gender Bias through LLM Conditional Text Generation [64.79319733514266]
  Large Language Models (LLMs) can generate biased and toxic responses. We propose a conditional text generation mechanism without the need for predefined gender phrases and stereotypes.
  arXiv  Detail & Related papers  (2023-11-01T05:31:46Z)
• "Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters [97.11173801187816]
  Large Language Models (LLMs) have recently emerged as an effective tool to assist individuals in writing various types of content. This paper critically examines gender biases in LLM-generated reference letters.
  arXiv  Detail & Related papers  (2023-10-13T16:12:57Z)
• Gender bias and stereotypes in Large Language Models [0.6882042556551611]
  This paper investigates Large Language Models' behavior with respect to gender stereotypes. We use a simple paradigm to test the presence of gender bias, building on but differing from WinoBias. Our contributions in this paper are as follows: (a) LLMs are 3-6 times more likely to choose an occupation that stereotypically aligns with a person's gender; (b) these choices align with people's perceptions better than with the ground truth as reflected in official job statistics; (d) LLMs ignore crucial ambiguities in sentence structure 95% of the time in our study items, but when explicitly prompted, they recognize
  arXiv  Detail & Related papers  (2023-08-28T22:32:05Z)
• Towards Understanding Gender-Seniority Compound Bias in Natural Language Generation [64.65911758042914]
  We investigate how seniority impacts the degree of gender bias exhibited in pretrained neural generation models. Our results show that GPT-2 amplifies bias by considering women as junior and men as senior more often than the ground truth in both domains. These results suggest that NLP applications built using GPT-2 may harm women in professional capacities.
  arXiv  Detail & Related papers  (2022-05-19T20:05:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.