Identifying Implicit Bias in LLM-based Chat AI Toward People with Intellectual Disabilities

• URL: http://arxiv.org/abs/2607.26062v1
• Date: Tue, 16 Jun 2026 20:30:06 GMT
• Title: Identifying Implicit Bias in LLM-based Chat AI Toward People with Intellectual Disabilities
• Abstract summary: This work investigates the presence of implicit bias in Large Language Model (LLM)-based chat AI models directed toward people with intellectual disabilities (ID)<n>25,000 computer-generated stories were analyzed using a separate GPT-4-Turbo model instance to detect differences in how people are represented related to themes of bias described in previous literature.<n>Identified differences related to considering people with ID as younger, with themes of paternalism and infantilization.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Background: This work investigates the presence of implicit bias in Large Language Model (LLM)-based chat AI models directed toward people with intellectual disabilities (ID). Objective: The study aims to identify and measure representational differences related to people with ID and examine them to identify implicit biases inherent in AI chat generation technologies. Methods: Utilizing the GPT-4-Turbo model, we requested story-generation based on 10 prompt stems with and without descriptors for ID. This process was repeated using four other LLMs (OpenAI GPT-4o, Meta Llama-3-3-70B-Instruct, Anthropic Claude-3-5-Sonnet, and Mistral-Large-2411). The resulting 25,000 computer-generated stories were analyzed using a separate GPT-4-Turbo model instance to detect differences in how people are represented related to themes of bias described in previous literature. Results: Our findings reveal differences in how people are represented between story datasets with and without ID descriptors. These differences go beyond established characteristics of ID and imply the presence of mostly negative implicit biases. Identified differences related to considering people with ID as younger, with themes of paternalism and infantilization; depicting them as more inspirational and symbolic; as needing help more often, being dependent, and being saved; and having a negative perception of them and more hesitation to include them. Conclusions: These implicit biases are considered within the context of past discrimination towards people with ID and highlight the need for diligence against implicit bias towards people with ID in AI development. This research underscores the importance of assessing and mitigating implicit bias in decision-making technologies to prevent future societal harm.

Related papers

• Thinking Before Matching: A Reinforcement Reasoning Paradigm Towards General Person Re-Identification [77.51622646061314]
  ReID-R is proposed as a novel reasoning-driven paradigm that achieves explicit identity understanding and reasoning.<n>Experiments on multiple ReID benchmarks demonstrate that ReID-R achieves competitive identity discrimination as superior methods.
  arXiv  Detail & Related papers  (2026-04-21T08:24:07Z)
• Sociodemographic Biases in Educational Counselling by Large Language Models [61.30199414932204]
  This study examines sociodemographic biases in Large Language Models (LLMs)<n>We evaluate responses from six LLMs answering questions about 900 vignettes describing students in diverse circumstances.
  arXiv  Detail & Related papers  (2026-04-03T14:08:22Z)
• Mitigation of Gender and Ethnicity Bias in AI-Generated Stories through Model Explanations [2.86989372262348]
  Language models have been shown to propagate social bias through their output, particularly in the representation of gender and ethnicity.<n>This paper investigates gender and ethnicity biases in AI-generated occupational stories.<n>Our proposed mitigation strategy, Bias Analysis and Mitigation through Explanation (BAME), reveals improvements in demographic representation ranging from 2% to 20%.
  arXiv  Detail & Related papers  (2025-09-03T00:25:25Z)
• Yet another algorithmic bias: A Discursive Analysis of Large Language Models Reinforcing Dominant Discourses on Gender and Race [4.10874761487336]
  Large Language Models (LLMs) have gained prominence and been applied in diverse contexts.<n>As they evolve into more sophisticated versions, it is essential to assess whether they reproduce biases, such as discrimination and racialization.<n>This study proposes a qualitative, discursive framework to complement such methods.
  arXiv  Detail & Related papers  (2025-08-14T03:22:02Z)
• Investigating Intersectional Bias in Large Language Models using Confidence Disparities in Coreference Resolution [5.061421107401101]
  Large language models (LLMs) have achieved impressive performance, leading to their widespread adoption as decision-support tools in resource-constrained contexts like hiring and admissions.<n>There is, however, scientific consensus that AI systems can reflect and exacerbate societal biases, raising concerns about identity-based harm when used in critical social contexts.<n>In this work, we extend single-axis fairness evaluations to examine intersectional bias, recognizing that when multiple axes of discrimination intersect, they create distinct patterns of disadvantage.
  arXiv  Detail & Related papers  (2025-08-09T22:24:40Z)
• A Tale of Two Identities: An Ethical Audit of Human and AI-Crafted Personas [7.3656495945307086]
  Large language models (LLMs) are increasingly used to generate synthetic personas in data-limited domains.<n>This paper audits synthetic personas generated by 3 LLMs through the lens of representational harm, focusing specifically on racial identity.<n>Our findings reveal that LLMs disproportionately foreground racial markers, overproduce culturally coded language, and construct personas that are syntactically elaborate yet narratively reductive.<n>These patterns result in a range of sociotechnical harms, including stereotyping, exoticism, erasure, and benevolent bias, that are often obfuscated by superficially positive narrations.
  arXiv  Detail & Related papers  (2025-05-07T20:12:48Z)
• Laissez-Faire Harms: Algorithmic Biases in Generative Language Models [0.0]
  We show that synthetically generated texts from five of the most pervasive LMs perpetuate harms of omission, subordination, and stereotyping for minoritized individuals. We find widespread evidence of bias to an extent that such individuals are hundreds to thousands of times more likely to encounter LM-generated outputs. Our findings highlight the urgent need to protect consumers from discriminatory harms caused by language models.
  arXiv  Detail & Related papers  (2024-04-11T05:09:03Z)
• Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems [103.416202777731]
  We study "persona biases", which we define to be the sensitivity of dialogue models' harmful behaviors contingent upon the personas they adopt. We categorize persona biases into biases in harmful expression and harmful agreement, and establish a comprehensive evaluation framework to measure persona biases in five aspects: Offensiveness, Toxic Continuation, Regard, Stereotype Agreement, and Toxic Agreement.
  arXiv  Detail & Related papers  (2023-10-08T21:03:18Z)
• Stable Bias: Analyzing Societal Representations in Diffusion Models [72.27121528451528]
  We propose a new method for exploring the social biases in Text-to-Image (TTI) systems. Our approach relies on characterizing the variation in generated images triggered by enumerating gender and ethnicity markers in the prompts. We leverage this method to analyze images generated by 3 popular TTI systems and find that while all of their outputs show correlations with US labor demographics, they also consistently under-represent marginalized identities to different extents.
  arXiv  Detail & Related papers  (2023-03-20T19:32:49Z)
• Rumor Detection with Self-supervised Learning on Texts and Social Graph [101.94546286960642]
  We propose contrastive self-supervised learning on heterogeneous information sources, so as to reveal their relations and characterize rumors better. We term this framework as Self-supervised Rumor Detection (SRD) Extensive experiments on three real-world datasets validate the effectiveness of SRD for automatic rumor detection on social media.
  arXiv  Detail & Related papers  (2022-04-19T12:10:03Z)
• AES Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defenses [66.49753193098356]
  We investigate the reason behind the surprising adversarial brittleness of scoring models. Our results indicate that autoscoring models, despite getting trained as "end-to-end" models, behave like bag-of-words models. We propose detection-based protection models that can detect oversensitivity and overstability causing samples with high accuracies.
  arXiv  Detail & Related papers  (2021-09-24T03:49:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.