A Tale of Two Identities: An Ethical Audit of Human and AI-Crafted Personas

• URL: http://arxiv.org/abs/2505.07850v1
• Date: Wed, 07 May 2025 20:12:48 GMT
• Title: A Tale of Two Identities: An Ethical Audit of Human and AI-Crafted Personas
• Authors: Pranav Narayanan Venkit, Jiayi Li, Yingfan Zhou, Sarah Rajtmajer, Shomir Wilson,
• Abstract summary: Large language models (LLMs) are increasingly used to generate synthetic personas in data-limited domains.<n>This paper audits synthetic personas generated by 3 LLMs through the lens of representational harm, focusing specifically on racial identity.<n>Our findings reveal that LLMs disproportionately foreground racial markers, overproduce culturally coded language, and construct personas that are syntactically elaborate yet narratively reductive.<n>These patterns result in a range of sociotechnical harms, including stereotyping, exoticism, erasure, and benevolent bias, that are often obfuscated by superficially positive narrations.
• Score: 7.3656495945307086
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: As LLMs (large language models) are increasingly used to generate synthetic personas particularly in data-limited domains such as health, privacy, and HCI, it becomes necessary to understand how these narratives represent identity, especially that of minority communities. In this paper, we audit synthetic personas generated by 3 LLMs (GPT4o, Gemini 1.5 Pro, Deepseek 2.5) through the lens of representational harm, focusing specifically on racial identity. Using a mixed methods approach combining close reading, lexical analysis, and a parameterized creativity framework, we compare 1512 LLM generated personas to human-authored responses. Our findings reveal that LLMs disproportionately foreground racial markers, overproduce culturally coded language, and construct personas that are syntactically elaborate yet narratively reductive. These patterns result in a range of sociotechnical harms, including stereotyping, exoticism, erasure, and benevolent bias, that are often obfuscated by superficially positive narrations. We formalize this phenomenon as algorithmic othering, where minoritized identities are rendered hypervisible but less authentic. Based on these findings, we offer design recommendations for narrative-aware evaluation metrics and community-centered validation protocols for synthetic identity generation.

Related papers

• Poor Alignment and Steerability of Large Language Models: Evidence from College Admission Essays [19.405531377930977]
  We investigate the use of large language models (LLM) in high-stakes admissions contexts.<n>We find that both types of LLM-generated essays are linguistically distinct from human-authored essays.<n>The demographically prompted and unprompted synthetic texts were also more similar to each other than to the human text.
  arXiv  Detail & Related papers  (2025-03-25T20:54:50Z)
• Inclusivity in Large Language Models: Personality Traits and Gender Bias in Scientific Abstracts [49.97673761305336]
  We evaluate three large language models (LLMs) for their alignment with human narrative styles and potential gender biases. Our findings indicate that, while these models generally produce text closely resembling human authored content, variations in stylistic features suggest significant gender biases.
  arXiv  Detail & Related papers  (2024-06-27T19:26:11Z)
• Measuring Psychological Depth in Language Models [50.48914935872879]
  We introduce the Psychological Depth Scale (PDS), a novel framework rooted in literary theory that measures an LLM's ability to produce authentic and narratively complex stories. We empirically validate our framework by showing that humans can consistently evaluate stories based on PDS (0.72 Krippendorff's alpha) Surprisingly, GPT-4 stories either surpassed or were statistically indistinguishable from highly-rated human-written stories sourced from Reddit.
  arXiv  Detail & Related papers  (2024-06-18T14:51:54Z)
• White Men Lead, Black Women Help? Benchmarking and Mitigating Language Agency Social Biases in LLMs [58.27353205269664]
  Social biases can manifest in language agency in Large Language Model (LLM)-generated content.<n>We introduce the Language Agency Bias Evaluation benchmark, which comprehensively evaluates biases in LLMs.<n>Using LABE, we unveil language agency social biases in 3 recent LLMs: ChatGPT, Llama3, and Mistral.
  arXiv  Detail & Related papers  (2024-04-16T12:27:54Z)
• Laissez-Faire Harms: Algorithmic Biases in Generative Language Models [0.0]
  We show that synthetically generated texts from five of the most pervasive LMs perpetuate harms of omission, subordination, and stereotyping for minoritized individuals. We find widespread evidence of bias to an extent that such individuals are hundreds to thousands of times more likely to encounter LM-generated outputs. Our findings highlight the urgent need to protect consumers from discriminatory harms caused by language models.
  arXiv  Detail & Related papers  (2024-04-11T05:09:03Z)
• Large language models that replace human participants can harmfully misportray and flatten identity groups [36.36009232890876]
  We show that there are two inherent limitations in the way current LLMs are trained that prevent this.<n>We argue analytically for why LLMs are likely to both misportray and flatten the representations of demographic groups.<n>We also discuss a third limitation about how identity prompts can essentialize identities.
  arXiv  Detail & Related papers  (2024-02-02T21:21:06Z)
• "Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters [97.11173801187816]
  Large Language Models (LLMs) have recently emerged as an effective tool to assist individuals in writing various types of content. This paper critically examines gender biases in LLM-generated reference letters.
  arXiv  Detail & Related papers  (2023-10-13T16:12:57Z)
• Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems [103.416202777731]
  We study "persona biases", which we define to be the sensitivity of dialogue models' harmful behaviors contingent upon the personas they adopt. We categorize persona biases into biases in harmful expression and harmful agreement, and establish a comprehensive evaluation framework to measure persona biases in five aspects: Offensiveness, Toxic Continuation, Regard, Stereotype Agreement, and Toxic Agreement.
  arXiv  Detail & Related papers  (2023-10-08T21:03:18Z)
• Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models [33.157279170602784]
  We present Marked Personas, a prompt-based method to measure stereotypes in large language models (LLMs) We find that portrayals generated by GPT-3.5 and GPT-4 contain higher rates of racial stereotypes than human-written portrayals using the same prompts. An intersectional lens reveals tropes that dominate portrayals of marginalized groups, such as tropicalism and the hypersexualization of minoritized women.
  arXiv  Detail & Related papers  (2023-05-29T16:29:22Z)
• Out of One, Many: Using Language Models to Simulate Human Samples [3.278541277919869]
  We show that the "algorithmic bias" within one such tool -- the GPT-3 language model -- is both fine-grained and demographically correlated. We create "silicon samples" by conditioning the model on thousands of socio-demographic backstories from real human participants.
  arXiv  Detail & Related papers  (2022-09-14T19:53:32Z)
• Towards Understanding and Mitigating Social Biases in Language Models [107.82654101403264]
  Large-scale pretrained language models (LMs) can be potentially dangerous in manifesting undesirable representational biases. We propose steps towards mitigating social biases during text generation. Our empirical results and human evaluation demonstrate effectiveness in mitigating bias while retaining crucial contextual information.
  arXiv  Detail & Related papers  (2021-06-24T17:52:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.