LLM-as-a-Demographic: Whom Sociodemographic Prompting Helps, and Whom It Hurts

• URL: http://arxiv.org/abs/2609.00222v1
• Date: Mon, 31 Aug 2026 18:33:15 GMT
• Title: LLM-as-a-Demographic: Whom Sociodemographic Prompting Helps, and Whom It Hurts
• Abstract summary: Sociodemographic prompting conditions the judge on an annotator's demographic profile to align its judgments with the corresponding group's.<n>We compare predicted label distributions of 23 open-weight LLMs on three subjective tasks against those of real annotator groups.<n>First, a judge prompted with no demographics is not perspective-neutral: models best reproduce the judgments of White, college-educated annotators.<n>Second, demographic conditioning is asymmetric: it moves the judge toward majority groups and away from minority groups, most strongly on offensiveness, where intersectional profiles amplify the harm.
• Score: 10.704307144706226
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are increasingly used as judges for subjective tasks, where annotators disagree and the relevant question is not only how accurate a judge is, but whose judgments it reproduces. Sociodemographic prompting conditions the judge on an annotator's demographic profile to align its judgments with the corresponding group's. We test whether this alignment emerges distributionally, comparing the predicted label distributions of 23 open-weight LLMs on three subjective tasks against those of real annotator groups, under three conditions: no demographic information, single-attribute profiles, and intersectional profiles over gender, age, race, and education. Three findings emerge. First, a judge prompted with no demographics is not perspective-neutral: models best reproduce the judgments of White, college-educated annotators. Second, demographic conditioning is asymmetric: it moves the judge toward majority groups and away from minority groups, most strongly on offensiveness, where intersectional profiles amplify the harm. Third, by comparing base and instruct models we identify instruction-tuning as a possible source of the asymmetry. Demographic conditioning should therefore be used with caution to estimate group judgments: conditioning moves predictions away from the reference distributions of the minority groups the method is often invoked to serve.

Related papers

• FairLens: Benchmarking Fairness in Vision-Language Models for High-Stakes Decision-Making [7.808503191719903]
  We evaluate vision-language models (VLMs) in three high-stakes domains: hiring, legal, and healthcare.<n>We find that the primary failure is unwarranted inference rather than unequal treatment.<n>We show that fair high-stakes VLM behavior requires similar treatment across groups and refusal to infer high-stakes attributes from appearance.
  arXiv  Detail & Related papers  (2026-09-01T16:06:50Z)
• FairFund-Bench: Evaluating Distributive Bias in LLM Resource Allocation [0.0]
  We introduce FairFund-Bench, a benchmark that systematically varies key features of previous audit designs.<n>We show that this disagreement can arise from differences in audit format and introduce FairFund-Bench.<n>The benchmark scores models on four criteria (demographic bias, deservingness alignment, cross-task consistency, and cross-context consistency) is publicly available and can be readily adapted to other substantive domains.
  arXiv  Detail & Related papers  (2026-07-31T01:29:16Z)
• From Self to Other: Evaluating Demographic Perspective-Taking in LLM Hate Speech Annotation [5.762370982168011]
  Persona-conditioned Large Language Models (models prompted to adopt a specific demographic identity) have been proposed as a way to simulate diverse perspectives at scale.<n>We evaluate three aspects of human social judgement: (i) whether personas from different groups disagree in human-like ways (inter-group disagreement), (ii) whether they become more sensitive when content targets their own identity (in-group sensitivity), and (iii) whether they can accurately predict how another group would react (vicarious prediction).<n>Our results show that no model consistently captures all three dimensions, and performance is highly model-dependent and does not emerge reliably from
  arXiv  Detail & Related papers  (2026-06-04T15:09:58Z)
• Sociodemographic Biases in Educational Counselling by Large Language Models [61.30199414932204]
  This study examines sociodemographic biases in Large Language Models (LLMs)<n>We evaluate responses from six LLMs answering questions about 900 vignettes describing students in diverse circumstances.
  arXiv  Detail & Related papers  (2026-04-03T14:08:22Z)
• Widespread Gender and Pronoun Bias in Moral Judgments Across LLMs [0.37331950863394847]
  Large language models (LLMs) are increasingly used to assess moral or ethical statements.<n>This work presents a controlled, sentence-level study of how grammatical person, number, and gender markers influence LLM moral classifications of fairness.
  arXiv  Detail & Related papers  (2026-03-13T22:42:37Z)
• Mitigating Subgroup Disparities in Multi-Label Speech Emotion Recognition: A Pseudo-Labeling and Unsupervised Learning Approach [53.824673312331626]
  Implicit Demography Inference (IDI) module uses k-means clustering to mitigate bias in Speech Emotion Recognition (SER)<n>Experiments show that pseudo-labeling IDI reduces subgroup disparities, improving fairness metrics by over 28%.<n>Unsupervised IDI yields more than a 4.6% improvement in fairness metrics with a drop of less than 3.6% in SER performance.
  arXiv  Detail & Related papers  (2025-05-20T14:50:44Z)
• GenderBias-\emph{VL}: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probing [72.0343083866144]
  This paper introduces the GenderBias-emphVL benchmark to evaluate occupation-related gender bias in Large Vision-Language Models. Using our benchmark, we extensively evaluate 15 commonly used open-source LVLMs and state-of-the-art commercial APIs. Our findings reveal widespread gender biases in existing LVLMs.
  arXiv  Detail & Related papers  (2024-06-30T05:55:15Z)
• White Men Lead, Black Women Help? Benchmarking and Mitigating Language Agency Social Biases in LLMs [58.27353205269664]
  Social biases can manifest in language agency in Large Language Model (LLM)-generated content.<n>We introduce the Language Agency Bias Evaluation benchmark, which comprehensively evaluates biases in LLMs.<n>Using LABE, we unveil language agency social biases in 3 recent LLMs: ChatGPT, Llama3, and Mistral.
  arXiv  Detail & Related papers  (2024-04-16T12:27:54Z)
• Leveraging Prototypical Representations for Mitigating Social Bias without Demographic Information [50.29934517930506]
  DAFair is a novel approach to address social bias in language models. We leverage prototypical demographic texts and incorporate a regularization term during the fine-tuning process to mitigate bias.
  arXiv  Detail & Related papers  (2024-03-14T15:58:36Z)
• Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLMs [13.744746481528711]
  Large Language Models (LLMs) are widely used to simulate human responses across diverse contexts.<n>We evaluate nine popular LLMs on their ability to understand demographic differences in two subjective judgment tasks: politeness and offensiveness.<n>We find that in zero-shot settings, most models' predictions for both tasks align more closely with labels from White participants than those from Asian or Black participants.
  arXiv  Detail & Related papers  (2023-11-16T10:02:24Z)
• Evaluating the Fairness of Discriminative Foundation Models in Computer Vision [51.176061115977774]
  We propose a novel taxonomy for bias evaluation of discriminative foundation models, such as Contrastive Language-Pretraining (CLIP) We then systematically evaluate existing methods for mitigating bias in these models with respect to our taxonomy. Specifically, we evaluate OpenAI's CLIP and OpenCLIP models for key applications, such as zero-shot classification, image retrieval and image captioning.
  arXiv  Detail & Related papers  (2023-10-18T10:32:39Z)
• Identifying biases in legal data: An algorithmic fairness perspective [3.8073142980733]
  We study the problem of identifying and measuring biases in large-scale legal case data from an algorithmic fairness perspective. Our approach utilizes two regression models: A baseline that represents the decisions of a "typical" judge as given by the data and a "fair" judge that applies one of three fairness concepts.
  arXiv  Detail & Related papers  (2021-09-21T04:05:12Z)
• Mitigating Face Recognition Bias via Group Adaptive Classifier [53.15616844833305]
  This work aims to learn a fair face representation, where faces of every group could be more equally represented. Our work is able to mitigate face recognition bias across demographic groups while maintaining the competitive accuracy.
  arXiv  Detail & Related papers  (2020-06-13T06:43:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.