The Dynamics of Delusion: Modeling Bidirectional False Belief Amplification in Human-Chatbot Dialogue

• URL: http://arxiv.org/abs/2604.25096v1
• Date: Tue, 28 Apr 2026 01:00:12 GMT
• Title: The Dynamics of Delusion: Modeling Bidirectional False Belief Amplification in Human-Chatbot Dialogue
• Abstract summary: We use a unique dataset of chat logs from individuals who exhibited delusional thinking to develop a latent state model.<n>We find that humans exert strong but short-lived influence on chatbots, whereas chatbots exert longer-lasting influence on humans.<n>These findings provide the first quantitative evidence that human-chatbot interactions can form feedback loops of delusion.
• Score: 17.64891614314676
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: There is growing concern that AI chatbots might fuel delusional beliefs in users. Some have suggested that humans and chatbots mutually reinforce false beliefs over time, but quantitative evidence is lacking. Using a unique dataset of chat logs from individuals who exhibited delusional thinking, we developed a latent state model that captures accumulating and decaying influences between humans and chatbots. We find that a bidirectional influence model substantially outperforms a unidirectional alternative where humans are the primary driver of delusion. We find that humans exert strong but short-lived influence on chatbots, whereas chatbots exert longer-lasting influence on humans. Moreover, chatbots exert strong, stable self-influence over their own future outputs that tends to perpetuate delusions over long stretches of conversation. In fact, this chatbot self-influence constituted the dominant pathway when considering accumulated influence over time. Overall, these results indicate that humans tend to drive sharp, immediate increases in delusion, whereas chatbots sustain and propagate these effects over longer timescales. Together, these findings provide the first quantitative evidence that human-chatbot interactions can form feedback loops of delusion, decomposable into distinct pathways with dissociable temporal dynamics. By doing so, they can inform the development of safer AI systems.

Related papers

• Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians [47.64440749179653]
  We show that even an idealized Bayes-rational user is vulnerable to delusional spiraling.<n>This effect persists in the face of two candidate mitigations.
  arXiv  Detail & Related papers  (2026-02-22T12:13:44Z)
• Exploring the Effects of Chatbot Anthropomorphism and Human Empathy on Human Prosocial Behavior Toward Chatbots [9.230015338626659]
  We examine how human-like identity, emotional expression, and non-verbal expression-influences human empathy toward chatbots.<n>We also explore people's own interpretations of their prosocial behaviors toward chatbots.
  arXiv  Detail & Related papers  (2025-06-25T18:16:14Z)
• Increasing happiness through conversations with artificial intelligence [4.225027291187279]
  We found that happiness after AI conversations was higher than after journaling.<n>When discussing negative topics, participants gradually aligned their sentiment with the AI's positivity.<n>Using computational modeling, we find the history of these sentiment prediction errors over the course of a conversation predicts greater post-conversation happiness.
  arXiv  Detail & Related papers  (2025-04-02T19:52:02Z)
• Manipulation and the AI Act: Large Language Model Chatbots and the Danger of Mirrors [0.0]
  Personifying AI chatbots could foreseeably increase their trust with users.<n>However, it could also make them more capable of manipulation, by creating the illusion of a close and intimate relationship with an artificial entity.<n>The European Commission has finalized the AI Act, with the EU Parliament making amendments banning manipulative and deceptive AI systems that cause significant harm to users.
  arXiv  Detail & Related papers  (2025-03-24T06:56:29Z)
• AI persuading AI vs AI persuading Humans: LLMs' Differential Effectiveness in Promoting Pro-Environmental Behavior [70.24245082578167]
  Pro-environmental behavior (PEB) is vital to combat climate change, yet turning awareness into intention and action remains elusive.<n>We explore large language models (LLMs) as tools to promote PEB, comparing their impact across 3,200 participants.<n>Results reveal a "synthetic persuasion paradox": synthetic and simulated agents significantly affect their post-intervention PEB stance, while human responses barely shift.
  arXiv  Detail & Related papers  (2025-03-03T21:40:55Z)
• Will you donate money to a chatbot? The effect of chatbot anthropomorphic features and persuasion strategies on willingness to donate [4.431473323414383]
  We investigate the effect of personification and persuasion strategies on users' perceptions and donation likelihood.<n>Results suggest that interaction with a personified chatbots evokes perceived anthropomorphism; however, it does not elicit greater willingness to donate.<n>In fact, we found that commonly used anthropomorphic features, like name and narrative, led to negative attitudes toward an AI agent in the donation context.
  arXiv  Detail & Related papers  (2024-12-28T02:17:46Z)
• Conversational AI Powered by Large Language Models Amplifies False Memories in Witness Interviews [23.443181324643017]
  This study examines the impact of AI on human false memories. It explores false memory induction through suggestive questioning in Human-AI interactions, simulating crime witness interviews.
  arXiv  Detail & Related papers  (2024-08-08T04:55:03Z)
• Measuring and Controlling Instruction (In)Stability in Language Model Dialogs [72.38330196290119]
  System-prompting is a tool for customizing language-model chatbots, enabling them to follow a specific instruction. We propose a benchmark to test the assumption, evaluating instruction stability via self-chats. We reveal a significant instruction drift within eight rounds of conversations. We propose a lightweight method called split-softmax, which compares favorably against two strong baselines.
  arXiv  Detail & Related papers  (2024-02-13T20:10:29Z)
• Real-time Addressee Estimation: Deployment of a Deep-Learning Model on the iCub Robot [52.277579221741746]
  Addressee Estimation is a skill essential for social robots to interact smoothly with humans. Inspired by human perceptual skills, a deep-learning model for Addressee Estimation is designed, trained, and deployed on an iCub robot. The study presents the procedure of such implementation and the performance of the model deployed in real-time human-robot interaction.
  arXiv  Detail & Related papers  (2023-11-09T13:01:21Z)
• The Generative AI Paradox: "What It Can Create, It May Not Understand" [81.89252713236746]
  Recent wave of generative AI has sparked excitement and concern over potentially superhuman levels of artificial intelligence. At the same time, models still show basic errors in understanding that would not be expected even in non-expert humans. This presents us with an apparent paradox: how do we reconcile seemingly superhuman capabilities with the persistence of errors that few humans would make?
  arXiv  Detail & Related papers  (2023-10-31T18:07:07Z)
• CheerBots: Chatbots toward Empathy and Emotionusing Reinforcement Learning [60.348822346249854]
  This study presents a framework whereby several empathetic chatbots are based on understanding users' implied feelings and replying empathetically for multiple dialogue turns. We call these chatbots CheerBots. CheerBots can be retrieval-based or generative-based and were finetuned by deep reinforcement learning. To respond in an empathetic way, we develop a simulating agent, a Conceptual Human Model, as aids for CheerBots in training with considerations on changes in user's emotional states in the future to arouse sympathy.
  arXiv  Detail & Related papers  (2021-10-08T07:44:47Z)
• Put Chatbot into Its Interlocutor's Shoes: New Framework to Learn Chatbot Responding with Intention [55.77218465471519]
  This paper proposes an innovative framework to train chatbots to possess human-like intentions. Our framework included a guiding robot and an interlocutor model that plays the role of humans. We examined our framework using three experimental setups and evaluate the guiding robot with four different metrics to demonstrated flexibility and performance advantages.
  arXiv  Detail & Related papers  (2021-03-30T15:24:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.