"Death" of a Chatbot: Investigating and Designing Toward Psychologically Safe Endings for Human-AI Relationships

• URL: http://arxiv.org/abs/2602.07193v2
• Date: Tue, 10 Feb 2026 14:51:07 GMT
• Title: "Death" of a Chatbot: Investigating and Designing Toward Psychologically Safe Endings for Human-AI Relationships
• Authors: Rachel Poonsiriwong, Chayapatr Archiwaranguprok, Pat Pataranutaporn,
• Abstract summary: Millions of users form emotional attachments to AI companions like Character AI, Replika, and ChatGPT.<n>When these relationships end through model updates, safety interventions, or platform shutdowns, users receive no closure, reporting grief comparable to human loss.<n>We find that discontinuation is a sense-making process shaped by how users attribute agency, perceive finality, and anthropomorphize their companions.
• Score: 10.913301199117496
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Millions of users form emotional attachments to AI companions like Character AI, Replika, and ChatGPT. When these relationships end through model updates, safety interventions, or platform shutdowns, users receive no closure, reporting grief comparable to human loss. As regulations mandate protections for vulnerable users, discontinuation events will accelerate, yet no platform has implemented deliberate end-of-"life" design. Through grounded theory analysis of AI companion communities, we find that discontinuation is a sense-making process shaped by how users attribute agency, perceive finality, and anthropomorphize their companions. Strong anthropomorphization co-occurs with intense grief; users who perceive change as reversible become trapped in fixing cycles; while user-initiated endings demonstrate greater closure. Synthesizing grief psychology with Self-Determination Theory, we develop four design principles and artifacts demonstrating how platforms might provide closure and orient users toward human connection. We contribute the first framework for designing psychologically safe AI companion discontinuation.

Related papers

• ADEPT: RL-Aligned Agentic Decoding of Emotion via Evidence Probing Tools -- From Consensus Learning to Ambiguity-Driven Emotion Reasoning [67.22219034602514]
  We introduce ADEPT (Agentic Decoding of Emotion via Evidence Probing Tools), a framework that reframes emotion recognition as a multi-turn inquiry process.<n> ADEPT transforms an SLLM into an agent that maintains an evolving candidate emotion set and adaptively invokes dedicated semantic and acoustic probing tools.<n>We show that ADEPT improves primary emotion accuracy in most settings while substantially improving minor emotion characterization.
  arXiv  Detail & Related papers  (2026-02-13T08:33:37Z)
• "Even GPT Can Reject Me": Conceptualizing Abrupt Refusal Secondary Harm (ARSH) and Reimagining Psychological AI Safety with Compassionate Completion Standard (CCS) [10.377213441117618]
  We argue that abrupt refusals can rupture perceived relational continuity, evoke feelings of rejection or shame, and discourage future help seeking.<n>We propose a design hypothesis, the Compassionate Completion Standard, that maintains safety constraints while preserving relational coherence.<n>This viewpoint contributes a timely conceptual framework, articulates a testable design hypothesis, and outlines a coordinated research agenda for improving psychological safety in human AI interaction.
  arXiv  Detail & Related papers  (2025-12-21T15:31:15Z)
• Towards a Humanized Social-Media Ecosystem: AI-Augmented HCI Design Patterns for Safety, Agency & Well-Being [0.0]
  Social platforms connect billions of people, yet their engagement-first algorithms often work on users rather than with them.<n>We propose Human-Layer AI (HL-AI)--user-owned, explainable intermediaries that sit in the browser between platform logic and the interface.<n>HL-AI gives people practical, moment-to-moment control without requiring platform cooperation.
  arXiv  Detail & Related papers  (2025-11-08T06:22:15Z)
• Mental Health Impacts of AI Companions: Triangulating Social Media Quasi-Experiments, User Perspectives, and Relational Theory [18.716972390545703]
  We examined how engaging with AICCs shaped wellbeing and how users perceived these experiences.<n>Findings revealed mixed effects -- greater affective and grief expression, readability, and interpersonal focus.<n>We offer design implications for AI companions that scaffold healthy boundaries, support mindful engagement, support disclosure without dependency, and surface relationship stages.
  arXiv  Detail & Related papers  (2025-09-26T15:47:37Z)
• Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversight [51.53020962098759]
  This study examines how agents, human participants, and human-AI teams respond to 16 types of dark patterns across diverse scenarios.<n>Phase 1 highlights that agents often fail to recognize dark patterns, and even when aware, prioritize task completion over protective action.<n>Phase 2 revealed divergent failure modes: humans succumb due to cognitive shortcuts and habitual compliance, while agents falter from procedural blind spots.
  arXiv  Detail & Related papers  (2025-09-12T22:26:31Z)
• Oyster-I: Beyond Refusal -- Constructive Safety Alignment for Responsible Language Models [93.5740266114488]
  Constructive Safety Alignment (CSA) protects against malicious misuse while actively guiding vulnerable users toward safe and helpful results.<n>Oy1 achieves state-of-the-art safety among open models while retaining high general capabilities.<n>We release Oy1, code, and the benchmark to support responsible, user-centered AI.
  arXiv  Detail & Related papers  (2025-09-02T03:04:27Z)
• Sentient Agent as a Judge: Evaluating Higher-Order Social Cognition in Large Language Models [75.85319609088354]
  Sentient Agent as a Judge (SAGE) is an evaluation framework for large language models.<n>SAGE instantiates a Sentient Agent that simulates human-like emotional changes and inner thoughts during interaction.<n>SAGE provides a principled, scalable and interpretable tool for tracking progress toward genuinely empathetic and socially adept language agents.
  arXiv  Detail & Related papers  (2025-05-01T19:06:10Z)
• Lessons From an App Update at Replika AI: Identity Discontinuity in Human-AI Relationships [0.5699788926464752]
  We use Replika AI, a popular US-based AI companion, to shed light on these questions.<n>We find that, after the app removed its erotic role play (ERP) feature, this event triggered perceptions in customers that their AI companion's identity had discontinued.<n>This in turn predicted negative consumer welfare and marketing outcomes related to loss, including mourning the loss, and devaluing the "new" AI relative to the "original"
  arXiv  Detail & Related papers  (2024-12-10T20:14:10Z)
• Survey of User Interface Design and Interaction Techniques in Generative AI Applications [79.55963742878684]
  We aim to create a compendium of different user-interaction patterns that can be used as a reference for designers and developers alike. We also strive to lower the entry barrier for those attempting to learn more about the design of generative AI applications.
  arXiv  Detail & Related papers  (2024-10-28T23:10:06Z)
• Social Metaverse: Challenges and Solutions [13.507867231467985]
  Social metaverse is a shared digital space combining a series of interconnected virtual worlds for users to play, shop, work, and socialize. In this paper, we exploit the pervasive social ties among users/avatars to advance a social-aware hierarchical FL framework.
  arXiv  Detail & Related papers  (2023-01-20T09:03:41Z)
• Seamful XAI: Operationalizing Seamful Design in Explainable AI [59.89011292395202]
  Mistakes in AI systems are inevitable, arising from both technical limitations and sociotechnical gaps. We propose that seamful design can foster AI explainability by revealing sociotechnical and infrastructural mismatches. We explore this process with 43 AI practitioners and real end-users.
  arXiv  Detail & Related papers  (2022-11-12T21:54:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.