STEREODISCO: Discovering Stereotypicality in LLMs

• URL: http://arxiv.org/abs/2607.27824v1
• Date: Thu, 30 Jul 2026 08:03:36 GMT
• Title: STEREODISCO: Discovering Stereotypicality in LLMs
• Abstract summary: We introduce STEREODISCO, a framework that adapts the semantic differential method to the systematic study of stereotypes in internal representations.<n>We find that the two LLMs agree with each other on social group ratings more than with humans.<n>We also discover stereotypical axes not investigated in prior work -- including humble vs. proud, narrow-minded vs. broad-minded, and cowardly vs. brave.
• Score: 14.61121805151903
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: LLMs encode, convey, and perpetuate stereotypes. Prior computational research focuses on a small set of semantic axes investigated in social psychology, and operates on word embeddings produced by language models, leaving open which other semantic axes carry stereotypical associations in LLMs and how LLMs internally represent such axes. We introduce STEREODISCO, a framework that adapts the semantic differential method (Osgood et al., 1957) to the systematic study of stereotypes in LLM internal representations. STEREODISCO constructs approx. 2,000 candidate semantic axes from WordNet antonym synsets, recovers each as a geometric axis in the LLM's activation space via probing, and identifies stereotypical axes via a statistical test over concept projections. As a case study, we apply STEREODISCO to social group stereotypes with LLAMA-3-8B-INSTRUCT and MISTRAL-7B-INSTRUCT. We find that the two LLMs agree with each other on social group ratings more than with humans, suggesting that LLM-encoded stereotype content diverges from that documented in social psychology. We also discover stereotypical axes not investigated in prior work -- including humble vs. proud, narrow-minded vs. broad-minded, and cowardly vs. brave, which human annotators independently confirm.

Related papers

• Measuring the Gap Between Human and LLM Research Ideas [63.27812052187884]
  We build an evaluation framework for ideation from high-quality human research papers.<n>For each paper, we reverse-engineer a small set of closely related prior works that likely inspired its core idea.<n>We introduce a two-axis research-taste taxonomy to profile each idea by its opportunity pattern and research paradigm.
  arXiv  Detail & Related papers  (2026-07-01T17:59:56Z)
• Free-form Association Tasks Reveal Stereotype Hallucination in Large Language Models [0.40418345173398346]
  We compare the structure of the stereotypes that humans and LLMs exhibit in the interpretation of free-form stimuli.<n>Humans engage in "stereotype exaggeration" by moderately amplifying first-order tendencies while preserving diversity.<n>By contrast, LLMs generate stark second-order stereotypes that neither amplify existing first-order tendencies nor reflect actual human group differences.
  arXiv  Detail & Related papers  (2026-06-29T21:58:18Z)
• Verbalizing LLMs' assumptions to explain and control sycophancy [62.927670321859495]
  LLMs can be socially sycophantic, affirming users when they ask questions like "am I in the wrong?" rather than providing genuine assessment.<n>We present Verbalized Assumptions, a framework for eliciting these assumptions from LLMs.<n>Our work contributes a new understanding of assumptions as a mechanism for sycophancy.
  arXiv  Detail & Related papers  (2026-04-03T14:15:43Z)
• Do Large Language Models Adapt to Language Variation across Socioeconomic Status? [29.1246345717672]
  Humans adjust their linguistic style to the audience they are addressing.<n>As these models increasingly mediate human-to-human communication, their failure to adapt to diverse styles can perpetuate stereotypes and marginalize communities.<n>We study the extent to which LLMs integrate into social media communication across different socioeconomic status (SES) communities.
  arXiv  Detail & Related papers  (2026-02-12T13:36:38Z)
• Simulating Identity, Propagating Bias: Abstraction and Stereotypes in LLM-Generated Text [8.988787218035414]
  We investigate whether persona-prompting leads to different levels of linguistic abstraction when generating short texts linking socio-demographic categories with stereotypical or non-stereotypical attributes.<n>Our results confirm criticisms about the ecology of personas as representative of socio-demographic groups and raise concerns about the risk of propagating stereotypes.
  arXiv  Detail & Related papers  (2025-09-10T10:49:21Z)
• How Deep is Love in LLMs' Hearts? Exploring Semantic Size in Human-like Cognition [75.11808682808065]
  This study investigates whether large language models (LLMs) exhibit similar tendencies in understanding semantic size.<n>Our findings reveal that multi-modal training is crucial for LLMs to achieve more human-like understanding.<n> Lastly, we examine whether LLMs are influenced by attention-grabbing headlines with larger semantic sizes in a real-world web shopping scenario.
  arXiv  Detail & Related papers  (2025-03-01T03:35:56Z)
• Large Language Models Reflect the Ideology of their Creators [71.65505524599888]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language.<n>This paper shows that the ideological stance of an LLM appears to reflect the worldview of its creators.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• Hate Personified: Investigating the role of LLMs in content moderation [64.26243779985393]
  For subjective tasks such as hate detection, where people perceive hate differently, the Large Language Model's (LLM) ability to represent diverse groups is unclear. By including additional context in prompts, we analyze LLM's sensitivity to geographical priming, persona attributes, and numerical information to assess how well the needs of various groups are reflected.
  arXiv  Detail & Related papers  (2024-10-03T16:43:17Z)
• A Taxonomy of Stereotype Content in Large Language Models [4.4212441764241]
  This study introduces a taxonomy of stereotype content in contemporary large language models (LLMs) We identify 14 stereotype dimensions (e.g., Morality, Ability, Health, Beliefs, Emotions) accounting for 90% of LLM stereotype associations. Our findings suggest that high-dimensional human stereotypes are reflected in LLMs and must be considered in AI auditing and debiasing to minimize unidentified harms.
  arXiv  Detail & Related papers  (2024-07-31T21:14:41Z)
• Beyond Human Norms: Unveiling Unique Values of Large Language Models through Interdisciplinary Approaches [69.73783026870998]
  This work proposes a novel framework, ValueLex, to reconstruct Large Language Models' unique value system from scratch. Based on Lexical Hypothesis, ValueLex introduces a generative approach to elicit diverse values from 30+ LLMs. We identify three core value dimensions, Competence, Character, and Integrity, each with specific subdimensions, revealing that LLMs possess a structured, albeit non-human, value system.
  arXiv  Detail & Related papers  (2024-04-19T09:44:51Z)
• StereoMap: Quantifying the Awareness of Human-like Stereotypes in Large Language Models [11.218531873222398]
  Large Language Models (LLMs) have been observed to encode and perpetuate harmful associations present in the training data. We propose a theoretically grounded framework called StereoMap to gain insights into their perceptions of how demographic groups have been viewed by society.
  arXiv  Detail & Related papers  (2023-10-20T17:22:30Z)
• Event knowledge in large language models: the gap between the impossible and the unlikely [46.540380831486125]
  We show that pre-trained large language models (LLMs) possess substantial event knowledge. They almost always assign higher likelihood to possible vs. impossible events. However, they show less consistent preferences for likely vs. unlikely events.
  arXiv  Detail & Related papers  (2022-12-02T23:43:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.