Related papers: The Role of Implicit and Explicit Demographic Signals in Large Language Model-based Student Assessment

The Role of Implicit and Explicit Demographic Signals in Large Language Model-based Student Assessment

URL: http://arxiv.org/abs/2609.16993v1
Date: Tue, 15 Sep 2026 11:07:26 GMT
Title: The Role of Implicit and Explicit Demographic Signals in Large Language Model-based Student Assessment
Abstract summary: Large Language Models are now common in student assessment, but we know little about how student demographics affect their use.<n>We set up controlled prompts to test 1) explicit demographic effects, where we mention demographic details directly, and 2) implicit effects, where we use conversation history as a demographic signal.<n>In both explicit and implicit cases, the models pick up on demographic cues and can change their scoring, feedback, and answers accordingly.
Score: 24.32015609344653
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Abstract: Large Language Models are now common in student assessment, but we know little about how student demographics affect their use. Sometimes, considering student demographics may be necessary -- for example, to improve readability for users with lower educational levels. However, it also risks being a cause of discrimination, e.g., when assigning lower scores to students from lower socioeconomic backgrounds. We set up controlled prompts to test 1) explicit demographic effects, where we mention demographic details directly, and 2) implicit effects, where we use conversation history as a demographic signal. We test these settings in three tasks: Automated Essay Scoring, Formative Feedback, and Metalinguistic Question Answering. We test six state-of-the-art LLMs on these tasks. In both explicit and implicit cases, the models pick up on demographic cues and can change their scoring, feedback, and answers accordingly. We find that LLMs frequently adjust the readability of feedback to education levels when these are explicitly mentioned. On the other hand, implicit conditions produce unpredictable biases, such as in question answering, where responses from lower-education levels receive lower sentiment scores. Our results provide clear evidence of demographic sensitivity in LLMs for educational assessment tasks.

Related papers

Large Language Models in K-12 Education: Alignment with State Curriculum Standards and Student Personas [3.147897243459717]
We identify variations in U.S. History curricula across states and evaluate the extent to which different LLMs reflect these state-specific curricular differences.<n>We find that while models are able to adjust their presentation of historical topics, these shifts may come from the perceived political leanings of states.
arXiv Detail & Related papers (2026-06-03T13:14:33Z)
Sociodemographic Biases in Educational Counselling by Large Language Models [61.30199414932204]
This study examines sociodemographic biases in Large Language Models (LLMs)<n>We evaluate responses from six LLMs answering questions about 900 vignettes describing students in diverse circumstances.
arXiv Detail & Related papers (2026-04-03T14:08:22Z)
DAIQ: Auditing Demographic Attribute Inference from Question in LLMs [3.1677998308405786]
Large Language Models (LLMs) are known to reflect social biases when demographic attributes, such as gender or race, are explicitly present in the input.<n>But even in their absence, these models still infer user identities based solely on question phrasing.<n>We introduce Demographic Attribute Inference from Questions (DAIQ), a task and framework for auditing an overlooked failure mode in language models.
arXiv Detail & Related papers (2025-08-18T19:26:17Z)
Revisiting LLM Value Probing Strategies: Are They Robust and Expressive? [81.49470136653665]
We evaluate the robustness and expressiveness of value representations across three widely used probing strategies.<n>We show that the demographic context has little effect on the free-text generation, and the models' values only weakly correlate with their preference for value-based actions.
arXiv Detail & Related papers (2025-07-17T18:56:41Z)
Does the Prompt-based Large Language Model Recognize Students' Demographics and Introduce Bias in Essay Scoring? [3.7498611358320733]
Large Language Models (LLMs) are widely used in Automated Essay Scoring (AES)<n>This study explores the relationship between the model's predictive power of students' demographic attributes based on their written works and its predictive bias in the scoring task in the prompt-based paradigm.
arXiv Detail & Related papers (2025-04-30T05:36:28Z)
Embracing AI in Education: Understanding the Surge in Large Language Model Use by Secondary Students [53.20318273452059]
Large language models (LLMs) like OpenAI's ChatGPT have opened up new avenues in education.<n>Despite school restrictions, our survey of over 300 middle and high school students revealed that a remarkable 70% of students have utilized LLMs.<n>We propose a few ideas to address such issues, including subject-specific models, personalized learning, and AI classrooms.
arXiv Detail & Related papers (2024-11-27T19:19:34Z)
Which Demographics do LLMs Default to During Annotation? [9.190535758368567]
Two research directions developed in the context of using large language models (LLM) for data annotations.<n>We evaluate which attributes of human annotators LLMs inherently mimic.<n>We observe notable influences related to gender, race, and age in demographic prompting.
arXiv Detail & Related papers (2024-10-11T14:02:42Z)
Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLMs [13.744746481528711]
Large Language Models (LLMs) are widely used to simulate human responses across diverse contexts.<n>We evaluate nine popular LLMs on their ability to understand demographic differences in two subjective judgment tasks: politeness and offensiveness.<n>We find that in zero-shot settings, most models' predictions for both tasks align more closely with labels from White participants than those from Asian or Black participants.
arXiv Detail & Related papers (2023-11-16T10:02:24Z)
Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs [67.51906565969227]
We study the unintended side-effects of persona assignment on the ability of LLMs to perform basic reasoning tasks. Our study covers 24 reasoning datasets, 4 LLMs, and 19 diverse personas (e.g. an Asian person) spanning 5 socio-demographic groups.
arXiv Detail & Related papers (2023-11-08T18:52:17Z)
Do LLMs exhibit human-like response biases? A case study in survey design [66.1850490474361]
We investigate the extent to which large language models (LLMs) reflect human response biases, if at all. We design a dataset and framework to evaluate whether LLMs exhibit human-like response biases in survey questionnaires. Our comprehensive evaluation of nine models shows that popular open and commercial LLMs generally fail to reflect human-like behavior.
arXiv Detail & Related papers (2023-11-07T15:40:43Z)
Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Prompting [64.80538055623842]
sociodemographic prompting is a technique that steers the output of prompt-based models towards answers that humans with specific sociodemographic profiles would give. We show that sociodemographic information affects model predictions and can be beneficial for improving zero-shot learning in subjective NLP tasks.
arXiv Detail & Related papers (2023-09-13T15:42:06Z)
Towards Controllable Biases in Language Generation [87.89632038677912]
We develop a method to induce societal biases in generated text when input prompts contain mentions of specific demographic groups. We analyze two scenarios: 1) inducing negative biases for one demographic and positive biases for another demographic, and 2) equalizing biases between demographics.
arXiv Detail & Related papers (2020-05-01T08:25:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.