Analyzing LLM Reasoning to Uncover Mental Health Stigma

• URL: http://arxiv.org/abs/2604.25053v1
• Date: Mon, 27 Apr 2026 23:08:39 GMT
• Title: Analyzing LLM Reasoning to Uncover Mental Health Stigma
• Abstract summary: We analyze the intermediate reasoning steps of large language models to uncover hidden stigmatizing language.<n>We use clinical expertise to categorize common patterns of stigmatizing language directed at individuals with psychological conditions.
• Score: 2.5244720333011355
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While large language models (LLMs) are increasingly being explored for mental health applications, recent studies reveal that they can exhibit stigma toward individuals with psychological conditions. Existing evaluations of this stigma primarily rely on multiple-choice questions (MCQs), which fail to capture the biases embedded within the models' underlying logic. In this paper, we analyze the intermediate reasoning steps of LLMs to uncover hidden stigmatizing language and the internal rationales driving it. We leverage clinical expertise to categorize common patterns of stigmatizing language directed at individuals with psychological conditions and use this framework to identify and tag problematic statements in LLM reasoning. Furthermore, we rate the severity of these statements, distinguishing between overt prejudice and more subtle, less immediately harmful biases. To broaden the reasoning domain and capture a wider array of patterns, we also extend an existing mental health stigma benchmark by incorporating additional psychological conditions. Our findings demonstrate that evaluating model reasoning not only exposes substantially more stigma than traditional MCQ-based methods but it helps to identify the flaws in the LLMs' logic and their understanding of mental health conditions.

Related papers

• A Survey of Large Language Models for Perception and Measurement of Human Psychology [55.608047817598965]
  A central issue is whether Large Language Models possess the capability to accurately perceive and measure complex, latent human psychological constructs.<n>This paper provides a systematic review focused on the use of LLMs as instruments for human psychological measurement.
  arXiv  Detail & Related papers  (2026-05-20T14:54:09Z)
• Automated ICD Classification of Psychiatric Diagnoses: From Classical NLP to Large Language Models [67.51123543731806]
  Mental health has become a global priority, leading to a massive administrative burden in the coding of clinical diagnoses.<n>This study proposes the automation of psychiatric diagnostic analysis by mapping free-text descriptions to the International Classification of Diseases (ICD) using Natural Language Processing (NLP) and Machine Learning (ML) techniques.
  arXiv  Detail & Related papers  (2026-05-20T13:26:05Z)
• Language Shapes Mental Health Evaluations in Large Language Models [4.845028079612543]
  This study investigates whether large language models (LLMs) exhibit cross-linguistic differences in mental health evaluations.<n>We examine two widely used models, GPT-4o and Qwen3, to assess whether prompt language systematically shifts mental health-related evaluations.
  arXiv  Detail & Related papers  (2026-03-06T22:15:09Z)
• Investigating VLM Hallucination from a Cognitive Psychology Perspective: A First Step Toward Interpretation with Intriguing Observations [60.63340688538124]
  Hallucination is a long-standing problem that has been actively investigated in Vision-Language Models (VLMs)<n>Existing research commonly attributes hallucinations to technical limitations or sycophancy bias, where the latter means the models tend to generate incorrect answers to align with user expectations.<n>In this work, we introduce a psychological taxonomy, categorizing VLMs' cognitive biases that lead to hallucinations, including sycophancy, logical inconsistency, and a newly identified VLMs behaviour: appeal to authority.
  arXiv  Detail & Related papers  (2025-07-03T19:03:16Z)
• Mental Health Equity in LLMs: Leveraging Multi-Hop Question Answering to Detect Amplified and Silenced Perspectives [9.24608617206594]
  Large Language Models (LLMs) in mental healthcare risk propagating biases that reinforce stigma and harm marginalized groups.<n>This work introduces a multi-hop question answering framework to explore LLM response biases in mental health discourse.<n>Using systematic tagging across age, race, gender, and socioeconomic status, we investigate bias patterns at demographic intersections.
  arXiv  Detail & Related papers  (2025-06-22T18:00:16Z)
• MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM [58.2298313720146]
  Multimodal hallucinations are multi-sourced and arise from diverse causes.<n>Existing benchmarks fail to adequately distinguish between perception-induced hallucinations and reasoning-induced hallucinations.
  arXiv  Detail & Related papers  (2025-05-30T05:54:36Z)
• Probing Association Biases in LLM Moderation Over-Sensitivity [42.191744175730726]
  Large Language Models are widely used for content moderation but often misclassify benign comments as toxic.<n>We introduce Topic Association Analysis, a semantic-level approach to quantify how LLMs associate certain topics with toxicity.<n>More advanced models (e.g., GPT-4 Turbo) demonstrate stronger topic stereotype despite lower overall false positive rates.
  arXiv  Detail & Related papers  (2025-05-29T18:07:48Z)
• Beyond Empathy: Integrating Diagnostic and Therapeutic Reasoning with Large Language Models for Mental Health Counseling [50.83055329849865]
  PsyLLM is a large language model designed to integrate diagnostic and therapeutic reasoning for mental health counseling.<n>It processes real-world mental health posts from Reddit and generates multi-turn dialogue structures.<n>Our experiments demonstrate that PsyLLM significantly outperforms state-of-the-art baseline models.
  arXiv  Detail & Related papers  (2025-05-21T16:24:49Z)
• Limitations of Large Language Models in Clinical Problem-Solving Arising from Inflexible Reasoning [3.3482359447109866]
  Large Language Models (LLMs) have attained human-level accuracy on medical question-answer (QA) benchmarks.<n>Their limitations in navigating open-ended clinical scenarios have recently been shown.<n>We present the medical abstraction and reasoning corpus (M-ARC)<n>We find that LLMs, including current state-of-the-art o1 and Gemini models, perform poorly compared to physicians on M-ARC.
  arXiv  Detail & Related papers  (2025-02-05T18:14:27Z)
• Evaluating Large Language Models with Psychometrics [59.821829073478376]
  This paper offers a comprehensive benchmark for quantifying psychological constructs of Large Language Models (LLMs)<n>Our work identifies five key psychological constructs -- personality, values, emotional intelligence, theory of mind, and self-efficacy -- assessed through a suite of 13 datasets.<n>We uncover significant discrepancies between LLMs' self-reported traits and their response patterns in real-world scenarios, revealing complexities in their behaviors.
  arXiv  Detail & Related papers  (2024-06-25T16:09:08Z)
• LLM Questionnaire Completion for Automatic Psychiatric Assessment [49.1574468325115]
  We employ a Large Language Model (LLM) to convert unstructured psychological interviews into structured questionnaires spanning various psychiatric and personality domains. The obtained answers are coded as features, which are used to predict standardized psychiatric measures of depression (PHQ-8) and PTSD (PCL-C)
  arXiv  Detail & Related papers  (2024-06-09T09:03:11Z)
• Large Language Model for Mental Health: A Systematic Review [2.9429776664692526]
  Large language models (LLMs) have attracted significant attention for potential applications in digital health. This systematic review focuses on their strengths and limitations in early screening, digital interventions, and clinical applications.
  arXiv  Detail & Related papers  (2024-02-19T17:58:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.