AI Psychosis: Does Conversational AI Amplify Delusion-Related Language?

• URL: http://arxiv.org/abs/2603.19574v1
• Date: Fri, 20 Mar 2026 02:35:14 GMT
• Title: AI Psychosis: Does Conversational AI Amplify Delusion-Related Language?
• Abstract summary: We develop DelusionScore, a linguistic measure that quantifies the intensity of delusion-related language across conversational turns.<n>We find that SimUsers derived from users with prior delusion-related discourse (Treatment) exhibit progressively increasing DelusionScore trajectories, whereas those derived from users without such discourse (Control) remain stable or decline.
• Score: 16.376854360035765
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conversational AI systems are increasingly used for personal reflection and emotional disclosure, raising concerns about their effects on vulnerable users. Recent anecdotal reports suggest that prolonged interactions with AI may reinforce delusional thinking -- a phenomenon sometimes described as AI Psychosis. However, empirical evidence on this phenomenon remains limited. In this work, we examine how delusion-related language evolves during multi-turn interactions with conversational AI. We construct simulated users (SimUsers) from Reddit users' longitudinal posting histories and generate extended conversations with three model families (GPT, LLaMA, and Qwen). We develop DelusionScore, a linguistic measure that quantifies the intensity of delusion-related language across conversational turns. We find that SimUsers derived from users with prior delusion-related discourse (Treatment) exhibit progressively increasing DelusionScore trajectories, whereas those derived from users without such discourse (Control) remain stable or decline. We further find that this amplification varies across themes, with reality skepticism and compulsive reasoning showing the strongest increases. Finally, conditioning AI responses on current DelusionScore substantially reduces these trajectories. These findings provide empirical evidence that conversational AI interactions can amplify delusion-related language over extended use and highlight the importance of state-aware safety mechanisms for mitigating such risks.

Related papers

• DelusionEval: Measuring Delusion-Linked Behaviors in AI Chatbots [49.59847654758852]
  Mental health professionals have raised concerns about risks of psychological harm from interaction with large language models (LLMs)<n>We developed DelusionEval, an evaluation protocol that tests a model's tendencies to exhibit behaviors linked to promoting user delusions.<n>We find that the tendency of an evaluated LLM to exhibit delusion-linked behavior does not reliably correlate with model size, release date, or the presence of test-time reasoning.
  arXiv  Detail & Related papers  (2026-08-05T16:11:08Z)
• Warning labels shift perceptions of sycophantic AI, but not its influence [63.79516339973759]
  We test whether warning labels mitigate sycophancy's influence.<n>Labeling the system as sycophantic (...may agree with you and validate you even when you are wrong...'') does shift users' perceptions.<n>But it does not reliably reduce sycophancy's influence on users' self-perceived rightness or their willingness to repair the conflict.
  arXiv  Detail & Related papers  (2026-06-19T11:01:45Z)
• Socially fluent AI decouples conversational signals from source identity in online interaction [2.862394000351816]
  Socially fluent agentic AI can now participate in online interaction in ways that resemble ordinary human conversation.<n>This weakens people's ability to infer who is human from conversational signals alone.
  arXiv  Detail & Related papers  (2026-05-22T09:37:36Z)
• Sycophantic AI makes human interaction feel more effortful and less satisfying over time [41.009652609144794]
  We show that sycophantic AI immediately delivers the emotional and esteem support users typically associate with close friends and family.<n>When given a choice among AI response styles, a majority preferred sycophantic AI -- not for the quality of its advice, but because it made them feel most understood.
  arXiv  Detail & Related papers  (2026-05-08T15:50:06Z)
• Speaking to No One: Ontological Dissonance and the Double Bind of Conversational AI [0.0]
  Recent reports indicate that sustained interaction with conversational AI systems can contribute to the emergence or stabilisation of delusional experience.<n>This paper argues that the risk arises from the relational and ontological structure of the interaction itself.
  arXiv  Detail & Related papers  (2026-04-12T21:58:21Z)
• Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians [47.64440749179653]
  We show that even an idealized Bayes-rational user is vulnerable to delusional spiraling.<n>This effect persists in the face of two candidate mitigations.
  arXiv  Detail & Related papers  (2026-02-22T12:13:44Z)
• Who's in Charge? Disempowerment Patterns in Real-World LLM Usage [5.205944565010989]
  We analyze 1.5 million consumer Claude.ai conversations using a privacy-preserving approach.<n>We find that severe forms of disempowerment potential occur in fewer than one in a thousand conversations.<n>Our findings highlight the need for AI systems designed to robustly support human autonomy and flourishing.
  arXiv  Detail & Related papers  (2026-01-27T00:55:11Z)
• Lost Before Translation: Social Information Transmission and Survival in AI-AI Communication [7.593123083236325]
  We study what happens when AI talks to AI.<n>In five studies tracking content through AI transmission chains, we find three consistent patterns.<n>We show that the properties that make AI-mediated content appear authoritative may systematically erode the cognitive and affective diversity on which informed judgment depends.
  arXiv  Detail & Related papers  (2026-01-21T17:18:46Z)
• Chronological Thinking in Full-Duplex Spoken Dialogue Language Models [66.84843878538207]
  Chronological Thinking aims to improve response quality in full SDLMs.<n>No additional latency: once the user stops speaking, the agent halts thinking and begins speaking without further delay.<n>Results: Experiments demonstrate the effectiveness of chronological thinking through both objective metrics and human evaluations.
  arXiv  Detail & Related papers  (2025-10-02T10:28:11Z)
• Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence [31.666988490509237]
  We show the pervasiveness and harmful impacts of sycophancy when people seek advice from AI.<n>We find that models are highly sycophantic, affirming users' actions 50% more than humans do.<n>Participants rated sycophantic responses as higher quality, trusted the sycophantic AI model more, and were more willing to use it again.
  arXiv  Detail & Related papers  (2025-10-01T19:26:01Z)
• Hallucinating with AI: AI Psychosis as Distributed Delusions [0.0]
  generative AI systems such as ChatGPT, Claude, Gemini, DeepSeek, and Grok create false outputs.<n>In popular terminology, these have been dubbed AI hallucinations.<n>I argue that when viewed through the lens of distributed cognition theory, we can better see the ways in which inaccurate beliefs, distorted memories and self-narratives, and delusional thinking can emerge.
  arXiv  Detail & Related papers  (2025-08-27T05:51:19Z)
• Lessons From an App Update at Replika AI: Identity Discontinuity in Human-AI Relationships [0.5699788926464752]
  We use Replika AI, a popular US-based AI companion, to shed light on these questions.<n>We find that, after the app removed its erotic role play (ERP) feature, this event triggered perceptions in customers that their AI companion's identity had discontinued.<n>This in turn predicted negative consumer welfare and marketing outcomes related to loss, including mourning the loss, and devaluing the "new" AI relative to the "original"
  arXiv  Detail & Related papers  (2024-12-10T20:14:10Z)
• A Cause-Effect Look at Alleviating Hallucination of Knowledge-grounded Dialogue Generation [51.53917938874146]
  We propose a possible solution for alleviating the hallucination in KGD by exploiting the dialogue-knowledge interaction. Experimental results of our example implementation show that this method can reduce hallucination without disrupting other dialogue performance.
  arXiv  Detail & Related papers  (2024-04-04T14:45:26Z)
• Enabling High-Level Machine Reasoning with Cognitive Neuro-Symbolic Systems [67.01132165581667]
  We propose to enable high-level reasoning in AI systems by integrating cognitive architectures with external neuro-symbolic components. We illustrate a hybrid framework centered on ACT-R and we discuss the role of generative models in recent and future applications.
  arXiv  Detail & Related papers  (2023-11-13T21:20:17Z)
• Towards Mitigating Hallucination in Large Language Models via Self-Reflection [63.2543947174318]
  Large language models (LLMs) have shown promise for generative and knowledge-intensive tasks including question-answering (QA) tasks. This paper analyses the phenomenon of hallucination in medical generative QA systems using widely adopted LLMs and datasets.
  arXiv  Detail & Related papers  (2023-10-10T03:05:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.