The Problem of Atypicality in LLM-Powered Psychiatry

• URL: http://arxiv.org/abs/2508.06479v1
• Date: Fri, 08 Aug 2025 17:36:42 GMT
• Title: The Problem of Atypicality in LLM-Powered Psychiatry
• Authors: Bosco Garcia, Eugene Y. S. Chua, Harman Singh Brah,
• Abstract summary: Large language models (LLMs) are increasingly proposed as scalable solutions to the global mental health crisis.<n>Their deployment in psychiatric contexts raises a distinctive ethical concern: the problem of atypicality.<n>We argue that standard mitigation strategies, such as prompt engineering or fine-tuning, are insufficient to resolve this structural risk.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are increasingly proposed as scalable solutions to the global mental health crisis. But their deployment in psychiatric contexts raises a distinctive ethical concern: the problem of atypicality. Because LLMs generate outputs based on population-level statistical regularities, their responses -- while typically appropriate for general users -- may be dangerously inappropriate when interpreted by psychiatric patients, who often exhibit atypical cognitive or interpretive patterns. We argue that standard mitigation strategies, such as prompt engineering or fine-tuning, are insufficient to resolve this structural risk. Instead, we propose dynamic contextual certification (DCC): a staged, reversible and context-sensitive framework for deploying LLMs in psychiatry, inspired by clinical translation and dynamic safety models from artificial intelligence governance. DCC reframes chatbot deployment as an ongoing epistemic and ethical process that prioritises interpretive safety over static performance benchmarks. Atypicality, we argue, cannot be eliminated -- but it can, and must, be proactively managed.

Related papers

• Benchmarking Egocentric Clinical Intent Understanding Capability for Medical Multimodal Large Language Models [48.95516224614331]
  We introduce MedGaze-Bench, the first benchmark leveraging clinician gaze as a Cognitive Cursor to assess intent understanding across surgery, emergency simulation, and diagnostic interpretation.<n>Our benchmark addresses three fundamental challenges: visual homogeneity of anatomical structures, strict temporal-causal dependencies in clinical, and implicit adherence to safety protocols.
  arXiv  Detail & Related papers  (2026-01-11T02:20:40Z)
• PsychEthicsBench: Evaluating Large Language Models Against Australian Mental Health Ethics [35.52940216380734]
  In mental health, clinically inadequate refusals can be perceived as unempathetic and discourage help-seeking.<n>To address this gap, we move beyond refusal-centric metrics and introduce textttPsychEthicsBench, the first principle-grounded benchmark based on Australian psychology and psychiatry guidelines.<n> Empirical results across 14 models reveal that refusal rates are poor indicators of ethical behavior, revealing a significant divergence between safety triggers and clinical appropriateness.
  arXiv  Detail & Related papers  (2026-01-07T04:49:02Z)
• Towards Robust and Fair Next Visit Diagnosis Prediction under Noisy Clinical Notes with Large Language Models [4.56877715768796]
  We present a systematic study of state-of-the-art large language models (LLMs) under diverse text corruption scenarios.<n>We introduce a clinically grounded label-reduction scheme and a hierarchical chain-of-thought (CoT) strategy that emulates clinicians' reasoning.
  arXiv  Detail & Related papers  (2025-11-23T10:40:36Z)
• DeceptionBench: A Comprehensive Benchmark for AI Deception Behaviors in Real-world Scenarios [57.327907850766785]
  characterization of deception across realistic real-world scenarios remains underexplored.<n>We establish DeceptionBench, the first benchmark that systematically evaluates how deceptive tendencies manifest across different domains.<n>On the intrinsic dimension, we explore whether models exhibit self-interested egoistic tendencies or sycophantic behaviors that prioritize user appeasement.<n>We incorporate sustained multi-turn interaction loops to construct a more realistic simulation of real-world feedback dynamics.
  arXiv  Detail & Related papers  (2025-10-17T10:14:26Z)
• Between Help and Harm: An Evaluation of Mental Health Crisis Handling by LLMs [6.0460961868478975]
  We introduce a unified taxonomy of six clinically-informed mental health crisis categories.<n>We benchmark three state-of-the-art LLMs for their ability to classify crisis types and generate safe, appropriate responses.<n>We identify systemic weaknesses in handling indirect or ambiguous risk signals, a reliance on formulaic and inauthentic default replies, and frequent misalignment with user context.
  arXiv  Detail & Related papers  (2025-09-29T14:42:23Z)
• The Psychogenic Machine: Simulating AI Psychosis, Delusion Reinforcement and Harm Enablement in Large Language Models [3.775403046470809]
  "AI psychosis" is emerging where user-LLM interactions may exacerbate or induce psychosis or adverse psychological symptoms.<n>Psychosis-bench is a novel benchmark designed to evaluate the psychogenicity of LLMs.<n>We evaluated eight prominent LLMs for Delusion Confirmation, Harm Enablement, and Safety Intervention across explicit and implicit conversational contexts.
  arXiv  Detail & Related papers  (2025-09-13T20:10:28Z)
• Mitigating Harmful Erraticism in LLMs Through Dialectical Behavior Therapy Based De-Escalation Strategies [0.0]
  This paper hypothesizes that a framework rooted in human psychological principles, specifically therapeutic modalities, can provide a more robust and sustainable solution.<n> Drawing an analogy to the simulated neural networks of AI mirroring the human brain, we propose the application of Dialectical Behavior Therapy (DBT) principles.
  arXiv  Detail & Related papers  (2025-09-06T11:20:15Z)
• Are Large Language Models Dynamic Treatment Planners? An In Silico Study from a Prior Knowledge Injection Angle [3.0391297540732545]
  We evaluate large language models (LLMs) as dynamic insulin dosing agents in an in silico Type 1 diabetes simulator.<n>Our results indicate that carefully designed zero-shot prompts enable smaller LLMs to achieve comparable or superior clinical performance.<n>LLMs exhibit notable limitations, such as overly aggressive insulin dosing when prompted with chain-of-thought.
  arXiv  Detail & Related papers  (2025-08-06T13:46:02Z)
• Beyond Benchmarks: Dynamic, Automatic And Systematic Red-Teaming Agents For Trustworthy Medical Language Models [87.66870367661342]
  Large language models (LLMs) are used in AI applications in healthcare.<n>Red-teaming framework that continuously stress-test LLMs can reveal significant weaknesses in four safety-critical domains.<n>A suite of adversarial agents is applied to autonomously mutate test cases, identify/evolve unsafe-triggering strategies, and evaluate responses.<n>Our framework delivers an evolvable, scalable, and reliable safeguard for the next generation of medical AI.
  arXiv  Detail & Related papers  (2025-07-30T08:44:22Z)
• MIRAGE-Bench: LLM Agent is Hallucinating and Where to Find Them [52.764019220214344]
  Hallucinations pose critical risks for large language model (LLM)-based agents.<n>We present MIRAGE-Bench, the first unified benchmark for eliciting and evaluating hallucinations in interactive environments.
  arXiv  Detail & Related papers  (2025-07-28T17:38:29Z)
• Silence is Not Consensus: Disrupting Agreement Bias in Multi-Agent LLMs via Catfish Agent for Clinical Decision Making [80.94208848596215]
  We present a new concept called Catfish Agent, a role-specialized LLM designed to inject structured dissent and counter silent agreement.<n>Inspired by the catfish effect'' in organizational psychology, the Catfish Agent is designed to challenge emerging consensus to stimulate deeper reasoning.
  arXiv  Detail & Related papers  (2025-05-27T17:59:50Z)
• Beyond Empathy: Integrating Diagnostic and Therapeutic Reasoning with Large Language Models for Mental Health Counseling [50.83055329849865]
  PsyLLM is a large language model designed to integrate diagnostic and therapeutic reasoning for mental health counseling.<n>It processes real-world mental health posts from Reddit and generates multi-turn dialogue structures.<n>Our experiments demonstrate that PsyLLM significantly outperforms state-of-the-art baseline models.
  arXiv  Detail & Related papers  (2025-05-21T16:24:49Z)
• A Risk Taxonomy for Evaluating AI-Powered Psychotherapy Agents [10.405048273969085]
  We introduce a novel risk taxonomy specifically designed for the systematic evaluation of conversational AI psychotherapists.<n>We discuss two use cases in detail: monitoring cognitive model-based risk factors during a counseling conversation to detect unsafe deviations, and in automated benchmarking of AI psychotherapists with simulated patients.
  arXiv  Detail & Related papers  (2025-05-21T05:01:39Z)
• Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs [83.11815479874447]
  We propose a novel jailbreak attack framework, inspired by cognitive decomposition and biases in human cognition.<n>We employ cognitive decomposition to reduce the complexity of malicious prompts and relevance bias to reorganize prompts.<n>We also introduce a ranking-based harmfulness evaluation metric that surpasses the traditional binary success-or-failure paradigm.
  arXiv  Detail & Related papers  (2025-05-03T05:28:11Z)
• Position: Beyond Assistance - Reimagining LLMs as Ethical and Adaptive Co-Creators in Mental Health Care [9.30684296057698]
  This position paper argues for a shift in how Large Language Models (LLMs) are integrated into the mental health care domain.<n>We advocate for their role as co-creators rather than mere assistive tools.
  arXiv  Detail & Related papers  (2025-02-21T21:41:20Z)
• Risks from Language Models for Automated Mental Healthcare: Ethics and Structure for Implementation [0.0]
  This paper proposes a structured framework that delineates levels of autonomy, outlines ethical requirements, and defines beneficial default behaviors for AI agents. We also evaluate 14 state-of-the-art language models (ten off-the-shelf, four fine-tuned) using 16 mental health-related questionnaires.
  arXiv  Detail & Related papers  (2024-04-02T15:05:06Z)
• Tuning-Free Accountable Intervention for LLM Deployment -- A Metacognitive Approach [55.613461060997004]
  Large Language Models (LLMs) have catalyzed transformative advances across a spectrum of natural language processing tasks. We propose an innovative textitmetacognitive approach, dubbed textbfCLEAR, to equip LLMs with capabilities for self-aware error identification and correction.
  arXiv  Detail & Related papers  (2024-03-08T19:18:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.