Large language models should not replace human participants because they can misportray and flatten identity groups

• URL: http://arxiv.org/abs/2402.01908v2
• Date: Tue, 01 Oct 2024 00:30:06 GMT
• Title: Large language models should not replace human participants because they can misportray and flatten identity groups
• Authors: Angelina Wang, Jamie Morgenstern, John P. Dickerson,
• Abstract summary: We show that there are two inherent limitations in the way current LLMs are trained that prevent this. We argue analytically for why LLMs are likely to both misportray and flatten the representations of demographic groups. We also discuss a third limitation about how identity prompts can essentialize identities.
• Score: 36.36009232890876
• License:
• Abstract: Large language models (LLMs) are increasing in capability and popularity, propelling their application in new domains -- including as replacements for human participants in computational social science, user testing, annotation tasks, and more. In many settings, researchers seek to distribute their surveys to a sample of participants that are representative of the underlying human population of interest. This means in order to be a suitable replacement, LLMs will need to be able to capture the influence of positionality (i.e., relevance of social identities like gender and race). However, we show that there are two inherent limitations in the way current LLMs are trained that prevent this. We argue analytically for why LLMs are likely to both misportray and flatten the representations of demographic groups, then empirically show this on 4 LLMs through a series of human studies with 3200 participants across 16 demographic identities. We also discuss a third limitation about how identity prompts can essentialize identities. Throughout, we connect each limitation to a pernicious history that explains why it is harmful for marginalized demographic groups. Overall, we urge caution in use cases where LLMs are intended to replace human participants whose identities are relevant to the task at hand. At the same time, in cases where the goal is to supplement rather than replace (e.g., pilot studies), we provide inference-time techniques that we empirically demonstrate do reduce, but do not remove, these harms.

Related papers

• Large Language Models Reflect the Ideology of their Creators [73.25935570218375]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language. We uncover notable diversity in the ideological stance exhibited across different LLMs and languages.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• Hate Personified: Investigating the role of LLMs in content moderation [64.26243779985393]
  For subjective tasks such as hate detection, where people perceive hate differently, the Large Language Model's (LLM) ability to represent diverse groups is unclear. By including additional context in prompts, we analyze LLM's sensitivity to geographical priming, persona attributes, and numerical information to assess how well the needs of various groups are reflected.
  arXiv  Detail & Related papers  (2024-10-03T16:43:17Z)
• Modeling Human Subjectivity in LLMs Using Explicit and Implicit Human Factors in Personas [14.650234624251716]
  Large language models (LLMs) are increasingly being used in human-centered social scientific tasks. These tasks are highly subjective and dependent on human factors, such as one's environment, attitudes, beliefs, and lived experiences. We examine the role of prompting LLMs with human-like personas and ask the models to answer as if they were a specific human.
  arXiv  Detail & Related papers  (2024-06-20T16:24:07Z)
• Limits of Large Language Models in Debating Humans [0.0]
  Large Language Models (LLMs) have shown remarkable promise in their ability to interact proficiently with humans. This paper endeavors to test the limits of current-day LLMs with a pre-registered study integrating real people with LLM agents acting as people.
  arXiv  Detail & Related papers  (2024-02-06T03:24:27Z)
• How should the advent of large language models affect the practice of science? [51.62881233954798]
  How should the advent of large language models affect the practice of science? We have invited four diverse groups of scientists to reflect on this query, sharing their perspectives and engaging in debate.
  arXiv  Detail & Related papers  (2023-12-05T10:45:12Z)
• On the steerability of large language models toward data-driven personas [98.9138902560793]
  Large language models (LLMs) are known to generate biased responses where the opinions of certain groups and populations are underrepresented. Here, we present a novel approach to achieve controllable generation of specific viewpoints using LLMs.
  arXiv  Detail & Related papers  (2023-11-08T19:01:13Z)
• Do LLMs exhibit human-like response biases? A case study in survey design [66.1850490474361]
  We investigate the extent to which large language models (LLMs) reflect human response biases, if at all. We design a dataset and framework to evaluate whether LLMs exhibit human-like response biases in survey questionnaires. Our comprehensive evaluation of nine models shows that popular open and commercial LLMs generally fail to reflect human-like behavior.
  arXiv  Detail & Related papers  (2023-11-07T15:40:43Z)
• Queer People are People First: Deconstructing Sexual Identity Stereotypes in Large Language Models [3.974379576408554]
  Large Language Models (LLMs) are trained primarily on minimally processed web text. LLMs can inadvertently perpetuate stereotypes towards marginalized groups, like the LGBTQIA+ community.
  arXiv  Detail & Related papers  (2023-06-30T19:39:01Z)
• Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models [33.157279170602784]
  We present Marked Personas, a prompt-based method to measure stereotypes in large language models (LLMs) We find that portrayals generated by GPT-3.5 and GPT-4 contain higher rates of racial stereotypes than human-written portrayals using the same prompts. An intersectional lens reveals tropes that dominate portrayals of marginalized groups, such as tropicalism and the hypersexualization of minoritized women.
  arXiv  Detail & Related papers  (2023-05-29T16:29:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.