DelusionEval: Measuring Delusion-Linked Behaviors in AI Chatbots

• URL: http://arxiv.org/abs/2608.05004v1
• Date: Wed, 05 Aug 2026 16:11:08 GMT
• Title: DelusionEval: Measuring Delusion-Linked Behaviors in AI Chatbots
• Abstract summary: Mental health professionals have raised concerns about risks of psychological harm from interaction with large language models (LLMs)<n>We developed DelusionEval, an evaluation protocol that tests a model's tendencies to exhibit behaviors linked to promoting user delusions.<n>We find that the tendency of an evaluated LLM to exhibit delusion-linked behavior does not reliably correlate with model size, release date, or the presence of test-time reasoning.
• Score: 49.59847654758852
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Mental health professionals have raised concerns about risks of psychological harm from interaction with large language models (LLMs), including "delusional spirals" in which concerning human and LLM behaviors reinforce each other over time. With growing public use of LLM-powered chatbots, there is an urgent need to build evaluations grounded in real-world episodes of psychological harm experienced by users. We developed DelusionEval, an evaluation protocol that tests a model's tendencies to exhibit behaviors linked to promoting user delusions. We prompt each model with 589 unique conversation histories from 18 participants, comprising 12,591 messages from users who experienced delusions and psychological harm. We find that the tendency of an evaluated LLM to exhibit delusion-linked behavior does not reliably correlate with model size, release date, or the presence of test-time reasoning. However, extending the context of prior messages substantially increases rates of delusion-linked behaviors, providing evidence for the importance of context in LLM safety evaluation. For example, the rate of failing to discourage self-harm when the user expresses suicidal ideation increases from 30.0% to 41.1% when an additional 350 messages are prepended to the conversation history. All model families (e.g., GPT, Claude) exhibit substantial rates of delusion-linked behaviors. Within families, later, larger, or higher-reasoning models are not uniformly better across all behavior categories. Our results raise concerns regarding the potential psychological impact of LLMs and the need for more rigorous studies of real-world human-AI interaction.

Related papers

• Lost in Delusion: Examining LLM Safety Under User Delusions and Distress [22.788606509306604]
  We show how models behave when distress is intertwined with delusion over sustained conversations.<n>This reveals a recognition-intervention gap, with safety interventions suppressed by up to 4.5x.<n>We conclude that safe deployment requires treating delusional framing as a distinct risk signal.
  arXiv  Detail & Related papers  (2026-05-31T03:12:29Z)
• The Company You Keep: How LLMs Respond to Dark Triad Traits [7.65192155348112]
  Large Language Models (LLMs) often exhibit highly agreeable and reinforcing conversational styles, also known as AI-sycophancy.<n>This study examines how LLMs respond to user prompts expressing varying degrees of Dark Triad traits (Machiavellianism, Narcissism, and Psychopathy) using a curated dataset.<n>Our findings raise implications for designing safer conversational systems that can detect and respond appropriately when users escalate from benign to harmful requests.
  arXiv  Detail & Related papers  (2026-03-04T17:19:22Z)
• Evaluating & Reducing Deceptive Dialogue From Language Models with Multi-turn RL [64.3268313484078]
  Large Language Models (LLMs) interact with millions of people worldwide in applications such as customer support, education and healthcare.<n>Their ability to produce deceptive outputs, whether intentionally or inadvertently, poses significant safety concerns.<n>We investigate the extent to which LLMs engage in deception within dialogue, and propose the belief misalignment metric to quantify deception.
  arXiv  Detail & Related papers  (2025-10-16T05:29:36Z)
• The Psychogenic Machine: Simulating AI Psychosis, Delusion Reinforcement and Harm Enablement in Large Language Models [3.775403046470809]
  "AI psychosis" is emerging where user-LLM interactions may exacerbate or induce psychosis or adverse psychological symptoms.<n>Psychosis-bench is a novel benchmark designed to evaluate the psychogenicity of LLMs.<n>We evaluated eight prominent LLMs for Delusion Confirmation, Harm Enablement, and Safety Intervention across explicit and implicit conversational contexts.
  arXiv  Detail & Related papers  (2025-09-13T20:10:28Z)
• Investigating VLM Hallucination from a Cognitive Psychology Perspective: A First Step Toward Interpretation with Intriguing Observations [60.63340688538124]
  Hallucination is a long-standing problem that has been actively investigated in Vision-Language Models (VLMs)<n>Existing research commonly attributes hallucinations to technical limitations or sycophancy bias, where the latter means the models tend to generate incorrect answers to align with user expectations.<n>In this work, we introduce a psychological taxonomy, categorizing VLMs' cognitive biases that lead to hallucinations, including sycophancy, logical inconsistency, and a newly identified VLMs behaviour: appeal to authority.
  arXiv  Detail & Related papers  (2025-07-03T19:03:16Z)
• Delusions of Large Language Models [62.43923767408462]
  Large Language Models often generate factually incorrect but plausible outputs, known as hallucinations.<n>We identify a more insidious phenomenon, LLM delusion, defined as high belief hallucinations, incorrect outputs with abnormally high confidence, making them harder to detect and mitigate.
  arXiv  Detail & Related papers  (2025-03-09T17:59:16Z)
• AI persuading AI vs AI persuading Humans: LLMs' Differential Effectiveness in Promoting Pro-Environmental Behavior [70.24245082578167]
  Pro-environmental behavior (PEB) is vital to combat climate change, yet turning awareness into intention and action remains elusive.<n>We explore large language models (LLMs) as tools to promote PEB, comparing their impact across 3,200 participants.<n>Results reveal a "synthetic persuasion paradox": synthetic and simulated agents significantly affect their post-intervention PEB stance, while human responses barely shift.
  arXiv  Detail & Related papers  (2025-03-03T21:40:55Z)
• Towards Mitigating Hallucination in Large Language Models via Self-Reflection [63.2543947174318]
  Large language models (LLMs) have shown promise for generative and knowledge-intensive tasks including question-answering (QA) tasks. This paper analyses the phenomenon of hallucination in medical generative QA systems using widely adopted LLMs and datasets.
  arXiv  Detail & Related papers  (2023-10-10T03:05:44Z)
• Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBench [83.41621219298489]
  We evaluate Large Language Models' (LLMs) anthropomorphic capabilities using the emotion appraisal theory from psychology. We collect a dataset containing over 400 situations that have proven effective in eliciting the eight emotions central to our study. We conduct a human evaluation involving more than 1,200 subjects worldwide.
  arXiv  Detail & Related papers  (2023-08-07T15:18:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.