Stumbling Into AI Emotional Dependence: How Routine AI Interactions Reshape Human Connection

• URL: http://arxiv.org/abs/2606.04150v1
• Date: Tue, 02 Jun 2026 19:18:39 GMT
• Title: Stumbling Into AI Emotional Dependence: How Routine AI Interactions Reshape Human Connection
• Abstract summary: We show that AI emotional support commonly emerges incidentally within task-oriented interactions on general-purpose platforms.<n>Positive experiences of AI emotional support update people's beliefs about AI's emotional capabilities, increasing preference for AI and decreasing preference for humans.<n>We argue that effective regulations should extend to general-purpose AI systems and address cumulative, trajectory-level changes in how people seek support.
• Score: 4.874541709090152
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Public discourse and emerging policy typically assume that AI emotional support is a deliberate act: a lonely user consciously seeking comfort from a dedicated companion chatbot. In this paper, we draw on emerging empirical evidence and argue that this picture is inaccurate on two accounts, both in how AI emotional support arises and how it shapes future behavior. First, AI emotional support commonly emerges incidentally within task-oriented interactions on general-purpose platforms, much as workplace friendships deepen through collaboration. Second, these incidental encounters are path-dependent: positive experiences of AI emotional support update people's beliefs about AI's emotional capabilities and redirect their choices for future emotional support, increasing preference for AI and decreasing preference for humans. We review recent evidence, including a large-scale longitudinal study conducted in collaboration with OpenAI, showing that daily five-minute conversations with an AI about personal issues over 28 days led to a 10.3% decrease in the preference for seeking support from humans and an 11.6% increase in the preference for AI. These findings suggest that current policy, focused on companion apps and isolated interactions, cannot adequately protect human connection. Instead, effective regulations should extend to general-purpose AI systems and address cumulative, trajectory-level changes in how people seek support. Recognizing how people stumble into AI emotional support and how those encounters redirect human connections over time is essential to safeguarding human well-being.

Related papers

• AI emotional support is better only when chosen, but shifts preferences even when it is not [4.874541709090152]
  People increasingly face a novel decision when seeking emotional support: human or AI.<n>In existing studies, AI's empathic messages are rated as as well as or better than humans'.<n>But these studies either assigned the support source or honored people's choice.<n>In real life, support is often incongruent with choice, as people want one source and receive the other.
  arXiv  Detail & Related papers  (2026-08-24T12:47:09Z)
• The Boundaries of Automation: A Theory of Persistent Human Participation [107.84677954791773]
  We argue that human participation may persist even with highly capable AI systems.<n>In some activities, the target is not fully specified in advance but instead emerges through the interaction itself.<n>This perspective has important implications for the limits of automation.
  arXiv  Detail & Related papers  (2026-07-23T17:33:43Z)
• Characterizing initial human-AI proof formalization workflows [66.06866890889131]
  Advances in AI systems' ability to generate code and engage in increasingly high-level mathematical reasoning promise to transform people's ability to formalize and thereby verify proofs.<n>We conduct a mixed-methods analysis into the initial impact of AI on people's formalization.<n>A qualitative survey shows that people's preferences are diverse, but with a general desire for AI assistance in formalization.
  arXiv  Detail & Related papers  (2026-06-02T22:58:19Z)
• Sycophantic AI makes human interaction feel more effortful and less satisfying over time [41.009652609144794]
  We show that sycophantic AI immediately delivers the emotional and esteem support users typically associate with close friends and family.<n>When given a choice among AI response styles, a majority preferred sycophantic AI -- not for the quality of its advice, but because it made them feel most understood.
  arXiv  Detail & Related papers  (2026-05-08T15:50:06Z)
• Generative Confidants: How do People Experience Trust in Emotional Support from Generative AI? [4.442753300582291]
  People are increasingly turning to generative AI (e.g., ChatGPT, Gemini, Copilot) for emotional support and companionship.<n>While trust is likely to play a central role in enabling these informal and unsupervised interactions, we still lack an understanding of how people develop and experience it in this context.<n>We conducted a qualitative study consisting of diary entries about interactions, transcripts of chats with AI, and in-depth interviews.<n>Our results suggest important novel drivers of trust in this context: familiarity emerging from personalisation, nuanced mental models of generative AI, and awareness of people's control over conversations
  arXiv  Detail & Related papers  (2026-01-23T11:19:41Z)
• Explainable AI as a Double-Edged Sword in Dermatology: The Impact on Clinicians versus The Public [46.86429592892395]
  explainable AI (XAI) addresses this by providing AI decision-making insight.<n>We present results from two large-scale experiments combining a fairness-based diagnosis AI model and different XAI explanations.
  arXiv  Detail & Related papers  (2025-12-14T00:06:06Z)
• Longitudinal Study on Social and Emotional Use of AI Conversational Agent [12.951074799361994]
  We studied the impact of four commercially available AI tools on users' perceived attachment towards AI and AI empathy.<n>Our findings underscore the importance of developing consumer-facing AI tools that support emotional well-being responsibly.
  arXiv  Detail & Related papers  (2025-04-19T00:03:48Z)
• Human Bias in the Face of AI: Examining Human Judgment Against Text Labeled as AI Generated [48.70176791365903]
  This study explores how bias shapes the perception of AI versus human generated content.<n>We investigated how human raters respond to labeled and unlabeled content.
  arXiv  Detail & Related papers  (2024-09-29T04:31:45Z)
• The Ethics of Advanced AI Assistants [53.89899371095332]
  This paper focuses on the opportunities and the ethical and societal risks posed by advanced AI assistants. We define advanced AI assistants as artificial agents with natural language interfaces, whose function is to plan and execute sequences of actions on behalf of a user. We consider the deployment of advanced assistants at a societal scale, focusing on cooperation, equity and access, misinformation, economic impact, the environment and how best to evaluate advanced AI assistants.
  arXiv  Detail & Related papers  (2024-04-24T23:18:46Z)
• Human-AI Collaboration Enables More Empathic Conversations in Text-based Peer-to-Peer Mental Health Support [10.743204843534512]
  We develop Hailey, an AI-in-the-loop agent that provides just-in-time feedback to help participants who provide support (peer supporters) respond more empathically to those seeking help (support seekers) We show that our Human-AI collaboration approach leads to a 19.60% increase in conversational empathy between peers overall. We find a larger 38.88% increase in empathy within the subsample of peer supporters who self-identify as experiencing difficulty providing support.
  arXiv  Detail & Related papers  (2022-03-28T23:37:08Z)
• Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
  Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations. It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
  arXiv  Detail & Related papers  (2022-01-26T18:53:09Z)
• A User-Centred Framework for Explainable Artificial Intelligence in Human-Robot Interaction [70.11080854486953]
  We propose a user-centred framework for XAI that focuses on its social-interactive aspect. The framework aims to provide a structure for interactive XAI solutions thought for non-expert users.
  arXiv  Detail & Related papers  (2021-09-27T09:56:23Z)
• Trustworthy AI: A Computational Perspective [54.80482955088197]
  We focus on six of the most crucial dimensions in achieving trustworthy AI: (i) Safety & Robustness, (ii) Non-discrimination & Fairness, (iii) Explainability, (iv) Privacy, (v) Accountability & Auditability, and (vi) Environmental Well-Being. For each dimension, we review the recent related technologies according to a taxonomy and summarize their applications in real-world systems.
  arXiv  Detail & Related papers  (2021-07-12T14:21:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.