Alignment by Stereotyping: How LLMs Sacrifice Individual Distinctiveness for Cultural Adaptation

• URL: http://arxiv.org/abs/2609.05993v1
• Date: Sat, 05 Sep 2026 09:33:08 GMT
• Title: Alignment by Stereotyping: How LLMs Sacrifice Individual Distinctiveness for Cultural Adaptation
• Abstract summary: We find that demographic profiles improve value alignment accuracy for most models, but at a systematic cost to individuality.<n>Top-performing models compress individuals far above the human baseline; within-family scaling amplifies this tradeoff while degrading intrinsic cultural understanding.<n>We further show that distributing demographic signals across conversational turns partially suppresses prototype retrieval compared to compact demographic labels.
• Score: 30.1403215719712
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models are increasingly deployed for personalized interaction, and demographic conditioning via user profiles is a widely adopted strategy for cultural adaptation. We ask whether this approach genuinely serves individual users or achieves accuracy by erasing individual distinctiveness. Studying seven models including frontier GPT-5.1 on the World Values Survey, we find that demographic profiles improve value alignment accuracy for most models, but at a systematic cost to individuality. That is, models pull responses toward demographic group centroids rather than preserving individual differences, a behavioral pattern we term alignment by stereotyping. Permutation tests (10,000 permutations, six demographic attributes, seven models) certify that top-performing models compress individuals far above the human baseline; within-family scaling amplifies this tradeoff while degrading intrinsic cultural understanding. Using a synthetic dialogue dataset validated on real human-chatbot conversations from PRISM (Kirk et al., 2024), we further show that distributing demographic signals across conversational turns partially suppresses prototype retrieval compared to compact demographic labels, a finding validated on real conversations via PRISM but requiring replication at larger scale.

Related papers

• Whose Norms? Disentangling Cultural and Personal Alignment in Large Language Models [66.34115263472931]
  Large language models are increasingly used for social decision-making situations that require balancing cultural norms with personal preferences.<n>We introduce PACT, the Personal-Preference and Cultural-Norm Trade-off framework, which evaluates whether models choose to follow a cultural norm or allow personal preferences.<n>Our five-country human study on PACT shows that culture-following in humans is mainly driven by scenario country, with the lowest agreement when participants judge their own cultural contexts.
  arXiv  Detail & Related papers  (2026-06-05T22:20:30Z)
• Learning Perspectivist Social Meaning via Demographic-Conditioned Fusion Embeddings [11.048086031066624]
  Social meaning in language inherently perspectival, varying across annotator backgrounds, demographics, and ideological positions.<n>Most NLP systems collapse this variation into a single ground-truth label, ignoring the diversity of interpretations.<n>We model social dimensions along a perspectivist spectrum, capturing how interpretations vary across demographic groups on a dataset consisting of 28k human annotations.
  arXiv  Detail & Related papers  (2026-06-05T10:25:01Z)
• Large Language Models Should Learn Personalized Rather Than Aggregated Human Preferences [2.276729420256043]
  We show that aggregation masks critical information about preference diversity, individual values, and contextual dependencies.<n>We conclude with a concrete research and policy agenda for developing preference-aware models.
  arXiv  Detail & Related papers  (2026-05-30T18:47:52Z)
• Can Persona-Prompted LLMs Emulate Subgroup Values? An Empirical Analysis of Generalisability and Fairness in Cultural Alignment [75.88231994139132]
  We show that even state-of-the-art models like GPT-4.1 achieve only 57.4% accuracy in predicting modal preferences.<n>We find significant pre-existing performance biases, where models better emulate young, male, Chinese, and Christian personas.
  arXiv  Detail & Related papers  (2026-04-14T15:06:13Z)
• Enhancing Personality Recognition by Comparing the Predictive Power of Traits, Facets, and Nuances [37.83859643892549]
  Personality recognition models aim to infer personality traits from different sources of behavioral data.<n>We trained a transformer-based model including cross-modal (audiovisual) and cross-subject (dyad-aware) attention mechanisms.<n>Results show that nuance-level models consistently outperform facet and trait-level models, reducing mean squared error by up to 74% across interaction scenarios.
  arXiv  Detail & Related papers  (2026-02-05T13:35:04Z)
• Measuring Social Bias in Vision-Language Models with Face-Only Counterfactuals from Real Photos [79.03150233804458]
  Real-world images entangle race and gender with correlated factors such as background and clothing, obscuring attribution.<n>We propose a textbfface-only counterfactual evaluation paradigm<n>We generate counterfactual variants by editing only facial attributes related to race and gender, keeping all other visual factors fixed.
  arXiv  Detail & Related papers  (2026-01-11T14:35:06Z)
• Can Language Models Reason about Individualistic Human Values and Preferences? [44.249817353449146]
  We study language models (LMs) on the challenge of individualistic value reasoning.<n>We find critical limitations in frontier LMs, which achieve only 55 % to 65% accuracy in predicting individualistic values.<n>We also identify a partiality of LMs in reasoning about global individualistic values, as measured by our proposed Value Inequity Index (sigmaInequity)
  arXiv  Detail & Related papers  (2024-10-04T19:03:41Z)
• On the steerability of large language models toward data-driven personas [98.9138902560793]
  Large language models (LLMs) are known to generate biased responses where the opinions of certain groups and populations are underrepresented. Here, we present a novel approach to achieve controllable generation of specific viewpoints using LLMs.
  arXiv  Detail & Related papers  (2023-11-08T19:01:13Z)
• Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems [103.416202777731]
  We study "persona biases", which we define to be the sensitivity of dialogue models' harmful behaviors contingent upon the personas they adopt. We categorize persona biases into biases in harmful expression and harmful agreement, and establish a comprehensive evaluation framework to measure persona biases in five aspects: Offensiveness, Toxic Continuation, Regard, Stereotype Agreement, and Toxic Agreement.
  arXiv  Detail & Related papers  (2023-10-08T21:03:18Z)
• Mitigating Face Recognition Bias via Group Adaptive Classifier [53.15616844833305]
  This work aims to learn a fair face representation, where faces of every group could be more equally represented. Our work is able to mitigate face recognition bias across demographic groups while maintaining the competitive accuracy.
  arXiv  Detail & Related papers  (2020-06-13T06:43:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.