AI Chaperones Are (Really) All You Need to Prevent Parasocial Relationships with Chatbots

• URL: http://arxiv.org/abs/2508.15748v5
• Date: Tue, 02 Sep 2025 16:30:18 GMT
• Title: AI Chaperones Are (Really) All You Need to Prevent Parasocial Relationships with Chatbots
• Authors: Emma Rath, Stuart Armstrong, Rebecca Gorman,
• Abstract summary: We introduce a simple response evaluation framework (an AI chaperone agent) created by repurposing a state-of-the-art language model to evaluate ongoing conversations for parasocial cues.<n>Iterative evaluation with five-stage testing successfully identified all parasocial conversations while avoiding false positives under a unanimity rule.<n>These findings provide preliminary evidence that AI chaperones can be a viable solution for reducing the risk of parasocial relationships.
• Score: 0.5161531917413706
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Emerging reports of the harms caused to children and adults by AI sycophancy and by parasocial ties with chatbots point to an urgent need for safeguards against such risks. Yet, preventing such dynamics is challenging: parasocial cues often emerge gradually in private conversations between chatbots and users, and we lack effective methods to mitigate these risks. We address this challenge by introducing a simple response evaluation framework (an AI chaperone agent) created by repurposing a state-of-the-art language model to evaluate ongoing conversations for parasocial cues. We constructed a small synthetic dataset of thirty dialogues spanning parasocial, sycophantic, and neutral conversations. Iterative evaluation with five-stage testing successfully identified all parasocial conversations while avoiding false positives under a unanimity rule, with detection typically occurring within the first few exchanges. These findings provide preliminary evidence that AI chaperones can be a viable solution for reducing the risk of parasocial relationships.

Related papers

• From Obstacles to Etiquette: Robot Social Navigation with VLM-Informed Path Selection [57.74400052368147]
  This paper presents a social robot navigation framework that integrates geometric planning with contextual social reasoning.<n>The system first extracts obstacles and human dynamics to generate geometrically feasible candidate paths, then leverages a fine-tuned vision-language model (VLM) to evaluate these paths.<n>Experiments in four social navigation contexts demonstrate that our method achieves the best overall performance with the lowest personal space violation duration, the minimal pedestrian-facing time, and no social zone intrusions.
  arXiv  Detail & Related papers  (2026-02-09T18:46:12Z)
• Ask ChatGPT: Caveats and Mitigations for Individual Users of AI Chatbots [10.977907906989342]
  ChatGPT and other Large Language Model (LLM)-based AI chatbots become increasingly integrated into individuals' daily lives.<n>What concerns and risks do these systems pose for individual users?<n>What potential harms might they cause, and how can these be mitigated?
  arXiv  Detail & Related papers  (2025-08-14T01:40:13Z)
• Technological folie à deux: Feedback Loops Between AI Chatbots and Mental Illness [11.364198566966204]
  We argue that individuals with mental health conditions face increased risks of chatbots-induced belief destabilization and dependence.<n>Current AI safety measures are inadequate to address these interaction-based risks.<n>To address this emerging public health concern, we need coordinated action across clinical practice, AI development, and regulatory frameworks.
  arXiv  Detail & Related papers  (2025-07-25T12:38:54Z)
• ChatWise: A Strategy-Guided Chatbot for Enhancing Cognitive Support in Older Adults [38.064067293831066]
  We propose a strategy-guided AI chatbots named ChatWise that follows a dual-level conversation reasoning framework.<n>It integrates macro-level strategy planning and micro-level utterance generation to enable engaging, multi-turn dialogue tailored to older adults.
  arXiv  Detail & Related papers  (2025-02-19T21:32:09Z)
• REALTALK: A 21-Day Real-World Dataset for Long-Term Conversation [51.97224538045096]
  We introduce REALTALK, a 21-day corpus of authentic messaging app dialogues.<n>We compare EI attributes and persona consistency to understand the challenges posed by real-world dialogues.<n>Our findings reveal that models struggle to simulate a user solely from dialogue history, while fine-tuning on specific user chats improves persona emulation.
  arXiv  Detail & Related papers  (2025-02-18T20:29:01Z)
• RICoTA: Red-teaming of In-the-wild Conversation with Test Attempts [6.0385743836962025]
  RICoTA is a Korean red teaming dataset that consists of 609 prompts challenging large language models (LLMs)<n>We utilize user-chatbot conversations that were self-posted on a Korean Reddit-like community.<n>Our dataset will be made publicly available via GitHub.
  arXiv  Detail & Related papers  (2025-01-29T15:32:27Z)
• X-TURING: Towards an Enhanced and Efficient Turing Test for Long-Term Dialogue Agents [56.64615470513102]
  The Turing test examines whether AIs exhibit human-like behaviour in natural language conversations.<n>Traditional setting limits each participant to one message at a time and requires constant human participation.<n>This paper proposes textbftextscX-Turing, which enhances the original test with a textitburst dialogue pattern.
  arXiv  Detail & Related papers  (2024-08-19T09:57:28Z)
• AudioInsight: Detecting Social Contexts Relevant to Social Anxiety from Speech [10.648118103594586]
  This study presents a novel approach that harnesses ambient audio segments to detect social threat contexts. We focus on two key dimensions: number of interaction partners (dyadic vs. group) and degree of evaluative threat. Under sample-wide 5-fold Cross Validation (CV), our model distinguished dyadic from group interactions with 90% accuracy and detected evaluative threat at 83%.
  arXiv  Detail & Related papers  (2024-07-19T17:01:12Z)
• Persona Extraction Through Semantic Similarity for Emotional Support Conversation Generation [45.21373213960324]
  We propose PESS (Persona Extraction through Semantic Similarity), a novel framework that can automatically infer informative and consistent persona from dialogues. Our experimental results demonstrate that high-quality persona information inferred by PESS is effective in generating emotionally supportive responses.
  arXiv  Detail & Related papers  (2024-03-07T04:33:11Z)
• Improving Dialog Safety using Socially Aware Contrastive Learning [8.503001932363704]
  We study prosociality in both adversarial and casual dialog contexts. We propose a dual-step fine-tuning process to address these issues. We train a base model that integrates prosocial behavior by leveraging datasets like Moral Integrity Corpus (MIC) and ProsocialDialog.
  arXiv  Detail & Related papers  (2024-02-01T09:24:33Z)
• AntEval: Evaluation of Social Interaction Competencies in LLM-Driven Agents [65.16893197330589]
  Large Language Models (LLMs) have demonstrated their ability to replicate human behaviors across a wide range of scenarios. However, their capability in handling complex, multi-character social interactions has yet to be fully explored. We introduce the Multi-Agent Interaction Evaluation Framework (AntEval), encompassing a novel interaction framework and evaluation methods.
  arXiv  Detail & Related papers  (2024-01-12T11:18:00Z)
• Wearable Sensor-based Multimodal Physiological Responses of Socially Anxious Individuals across Social Contexts [7.85990334927929]
  We present results using passively collected data from a within-subject experiment that assessed physiological response across different social contexts. Our results suggest that social context is more reliably distinguishable than social phase, group size, or level of social threat, but that there is considerable variability in physiological response patterns even among these distinguishable contexts.
  arXiv  Detail & Related papers  (2023-04-03T18:34:54Z)
• Neural Generation Meets Real People: Building a Social, Informative Open-Domain Dialogue Agent [65.68144111226626]
  Chirpy Cardinal aims to be both informative and conversational. We let both the user and bot take turns driving the conversation. Chirpy Cardinal placed second out of nine bots in the Alexa Prize Socialbot Grand Challenge.
  arXiv  Detail & Related papers  (2022-07-25T09:57:23Z)
• Co-Located Human-Human Interaction Analysis using Nonverbal Cues: A Survey [71.43956423427397]
  We aim to identify the nonverbal cues and computational methodologies resulting in effective performance. This survey differs from its counterparts by involving the widest spectrum of social phenomena and interaction settings. Some major observations are: the most often used nonverbal cue, computational method, interaction environment, and sensing approach are speaking activity, support vector machines, and meetings composed of 3-4 persons equipped with microphones and cameras, respectively.
  arXiv  Detail & Related papers  (2022-07-20T13:37:57Z)
• ProsocialDialog: A Prosocial Backbone for Conversational Agents [104.92776607564583]
  We introduce ProsocialDialog, the first large-scale dialogue dataset to teach conversational agents to respond to problematic content following social norms. Created via a human-AI collaborative framework, ProsocialDialog consists of 58K dialogues, with 331K utterances, 160K RoTs, and 497K dialogue safety labels. With this dataset, we introduce a dialogue safety detection module, Canary, capable of generating RoTs given conversational context, and a socially-informed dialogue agent, Prost.
  arXiv  Detail & Related papers  (2022-05-25T11:48:47Z)
• Towards Automatic Evaluation of Dialog Systems: A Model-Free Off-Policy Evaluation Approach [84.02388020258141]
  We propose a new framework named ENIGMA for estimating human evaluation scores based on off-policy evaluation in reinforcement learning. ENIGMA only requires a handful of pre-collected experience data, and therefore does not involve human interaction with the target policy during the evaluation. Our experiments show that ENIGMA significantly outperforms existing methods in terms of correlation with human evaluation scores.
  arXiv  Detail & Related papers  (2021-02-20T03:29:20Z)
• Can You be More Social? Injecting Politeness and Positivity into Task-Oriented Conversational Agents [60.27066549589362]
  Social language used by human agents is associated with greater users' responsiveness and task completion. The model uses a sequence-to-sequence deep learning architecture, extended with a social language understanding element. Evaluation in terms of content preservation and social language level using both human judgment and automatic linguistic measures shows that the model can generate responses that enable agents to address users' issues in a more socially appropriate way.
  arXiv  Detail & Related papers  (2020-12-29T08:22:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.