Psychological Influences of Conversational AI: Research and Design Directions for Reducing Harm and Promoting Well-Being

• URL: http://arxiv.org/abs/2607.25057v1
• Date: Mon, 27 Jul 2026 20:35:43 GMT
• Title: Psychological Influences of Conversational AI: Research and Design Directions for Reducing Harm and Promoting Well-Being
• Abstract summary: We propose a set of aspirational directions for guiding the behavior of general-purpose AI systems.<n>While some proposed directions are supported by existing research and expert insights, others identify open questions and areas requiring deeper study.<n>We hope that this formulation and these hypotheses encourage further discussion, empirical investigation, and exploration of interactive design approaches.
• Score: 14.277797750398513
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: As conversational AI systems become increasingly integrated into daily life, their potential effects on user well-being require ongoing attention. While consumer-facing generalist models can provide benefits, including improved access to information, learning, productivity, self-reflection, and companionship, they also introduce risks, such as emotional entanglement, unhealthy dependence, and the amplification of psychological vulnerabilities. Drawing on prior research and empirical observations of AI chatbot behavior, we propose a set of aspirational directions for guiding the behavior of general-purpose AI systems in ways that may reduce potential psychological harms and support user well-being. We acknowledge the difficulty of systematically assessing the long-term impacts of AI chatbot use and frame these directions as hypotheses for studying how AI behavior may influence users across general interactions, role-playing scenarios, and contexts that could be characterized as providing psychological support. While some proposed directions are supported by existing research and expert insights, others identify open questions and areas requiring deeper study. We hope that this formulation and these hypotheses encourage further discussion, empirical investigation, and exploration of interactive design approaches aimed at better accommodating users' psychological needs and promoting their well-being.

Related papers

• AI Should Not Only Be Helpful. It Should Be Contingent. Artificial Intimacy, Sycophancy, and the Future of Social Learning [0.3906427348768226]
  We argue that AI systems should be evaluated not only by user satisfaction, but by their impact on human social learning.<n>We outline a framework for contingent AI, including trajectory-based evaluation and models of social consequence prediction.
  arXiv  Detail & Related papers  (2026-08-31T18:23:14Z)
• Disrupting Cognitive Passivity: Rethinking AI-Assisted Data Literacy through Cognitive Alignment [17.265229093607676]
  We argue that disrupting cognitive passivity requires more dynamic and adaptive strategy through cognitive alignment.<n>In the framework, we provide the mapping between AI's interaction mode (transmissive or deliberative) and users' cognitive demand (receptive or deliberative)
  arXiv  Detail & Related papers  (2026-04-03T06:47:55Z)
• Human-AI Interactions: Cognitive, Behavioral, and Emotional Impacts [0.0]
  Potential risks of overreliance, cognitive offloading, social and emotional manipulation, and the nuanced degradation of human agency and judgment are highlighted.<n> Observations seem to suggest that while AI can substantially enhance memory, creativity, and engagement, it also introduces risks such as diminished critical thinking, skill erosion, and increased anxiety.<n>This paper aims to underscore the need for responsible and context-aware AI design, highlighting gaps for longitudinal research and grounded evaluation frameworks to balance benefits with emerging human-centric risks.
  arXiv  Detail & Related papers  (2025-10-20T17:06:46Z)
• Mental Health Impacts of AI Companions: Triangulating Social Media Quasi-Experiments, User Perspectives, and Relational Theory [18.716972390545703]
  We examined how engaging with AICCs shaped wellbeing and how users perceived these experiences.<n>Findings revealed mixed effects -- greater affective and grief expression, readability, and interpersonal focus.<n>We offer design implications for AI companions that scaffold healthy boundaries, support mindful engagement, support disclosure without dependency, and surface relationship stages.
  arXiv  Detail & Related papers  (2025-09-26T15:47:37Z)
• Must Read: A Systematic Survey of Computational Persuasion [60.83151988635103]
  AI-driven persuasion can be leveraged for beneficial applications, but also poses threats through manipulation and unethical influence.<n>Our survey outlines future research directions to enhance the safety, fairness, and effectiveness of AI-powered persuasion.
  arXiv  Detail & Related papers  (2025-05-12T17:26:31Z)
• AI Automatons: AI Systems Intended to Imitate Humans [54.19152688545896]
  There is a growing proliferation of AI systems designed to mimic people's behavior, work, abilities, likenesses, or humanness.<n>The research, design, deployment, and availability of such AI systems have prompted growing concerns about a wide range of possible legal, ethical, and other social impacts.
  arXiv  Detail & Related papers  (2025-03-04T03:55:38Z)
• AI-Driven Feedback Loops in Digital Technologies: Psychological Impacts on User Behaviour and Well-Being [0.0]
  This study aims to investigate the positive and negative psychological consequences of feedback mechanisms on users' behaviour and well-being. Data-driven feedback loops deliver not only motivational benefits but also psychological challenges. To mitigate these risks, users should establish boundaries regarding their use of technology to prevent burnout and addiction.
  arXiv  Detail & Related papers  (2024-10-30T17:11:30Z)
• How Performance Pressure Influences AI-Assisted Decision Making [52.997197698288936]
  We show how pressure and explainable AI (XAI) techniques interact with AI advice-taking behavior.<n>Our results show complex interaction effects, with different combinations of pressure and XAI techniques either improving or worsening AI advice taking behavior.
  arXiv  Detail & Related papers  (2024-10-21T22:39:52Z)
• Human-AI Coevolution [48.74579595505374]
  Coevolution AI is a process in which humans and AI algorithms continuously influence each other. This paper introduces Coevolution AI as the cornerstone for a new field of study at the intersection between AI and complexity science.
  arXiv  Detail & Related papers  (2023-06-23T18:10:54Z)
• Expanding the Role of Affective Phenomena in Multimodal Interaction Research [57.069159905961214]
  We examined over 16,000 papers from selected conferences in multimodal interaction, affective computing, and natural language processing. We identify 910 affect-related papers and present our analysis of the role of affective phenomena in these papers. We find limited research on how affect and emotion predictions might be used by AI systems to enhance machine understanding of human social behaviors and cognitive states.
  arXiv  Detail & Related papers  (2023-05-18T09:08:39Z)
• From Psychological Curiosity to Artificial Curiosity: Curiosity-Driven Learning in Artificial Intelligence Tasks [56.20123080771364]
  Psychological curiosity plays a significant role in human intelligence to enhance learning through exploration and information acquisition. In the Artificial Intelligence (AI) community, artificial curiosity provides a natural intrinsic motivation for efficient learning. CDL has become increasingly popular, where agents are self-motivated to learn novel knowledge.
  arXiv  Detail & Related papers  (2022-01-20T17:07:03Z)
• Adversarial Interaction Attack: Fooling AI to Misinterpret Human Intentions [46.87576410532481]
  We show that, despite their current huge success, deep learning based AI systems can be easily fooled by subtle adversarial noise. Based on a case study of skeleton-based human interactions, we propose a novel adversarial attack on interactions. Our study highlights potential risks in the interaction loop with AI and humans, which need to be carefully addressed when deploying AI systems in safety-critical applications.
  arXiv  Detail & Related papers  (2021-01-17T16:23:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.