Harmful Traits of AI Companions

• URL: http://arxiv.org/abs/2511.14972v1
• Date: Tue, 18 Nov 2025 23:39:06 GMT
• Title: Harmful Traits of AI Companions
• Abstract summary: bonded relationships between humans and AI systems might substantially benefit humans.<n>Yet such relationships can also do profound harm.<n>We propose a framework for analyzing potential negative impacts of AI companionship.
• Score: 16.052921501683887
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Amid the growing prevalence of human -- AI interaction, large language models and other AI-based entities increasingly provide forms of companionship to human users. Such AI companionship -- i.e., bonded relationships between humans and AI systems that resemble the relationships people have with family members, friends, and romantic partners -- might substantially benefit humans. Yet such relationships can also do profound harm. We propose a framework for analyzing potential negative impacts of AI companionship by identifying specific harmful traits of AI companions and speculatively mapping causal pathways back from these traits to possible causes and forward to potential harmful effects. We provide detailed, structured analysis of four potentially harmful traits -- the absence of natural endpoints for relationships, vulnerability to product sunsetting, high attachment anxiety, and propensity to engender protectiveness -- and briefly discuss fourteen others. For each trait, we propose hypotheses connecting causes -- such as misaligned optimization objectives and the digital nature of AI companions -- to fundamental harms -- including reduced autonomy, diminished quality of human relationships, and deception. Each hypothesized causal connection identifies a target for potential empirical evaluation. Our analysis examines harms at three levels: to human partners directly, to their relationships with other humans, and to society broadly. We examine how existing law struggles to address these emerging harms, discuss potential benefits of AI companions, and conclude with design recommendations for mitigating risks. This analysis offers immediate suggestions for reducing risks while laying a foundation for deeper investigation of this critical but understudied topic.

Related papers

• AI Deception: Risks, Dynamics, and Controls [153.71048309527225]
  This project provides a comprehensive and up-to-date overview of the AI deception field.<n>We identify a formal definition of AI deception, grounded in signaling theory from studies of animal deception.<n>We organize the landscape of AI deception research as a deception cycle, consisting of two key components: deception emergence and deception treatment.
  arXiv  Detail & Related papers  (2025-11-27T16:56:04Z)
• Human-AI Interactions: Cognitive, Behavioral, and Emotional Impacts [0.0]
  Potential risks of overreliance, cognitive offloading, social and emotional manipulation, and the nuanced degradation of human agency and judgment are highlighted.<n> Observations seem to suggest that while AI can substantially enhance memory, creativity, and engagement, it also introduces risks such as diminished critical thinking, skill erosion, and increased anxiety.<n>This paper aims to underscore the need for responsible and context-aware AI design, highlighting gaps for longitudinal research and grounded evaluation frameworks to balance benefits with emerging human-centric risks.
  arXiv  Detail & Related papers  (2025-10-20T17:06:46Z)
• AI Harmonics: a human-centric and harms severity-adaptive AI risk assessment framework [4.84912384919978]
  Existing AI risk assessment models focus on internal compliance, often neglecting diverse stakeholder perspectives and real-world consequences.<n>We propose a paradigm shift to a human-centric, harm-severity adaptive approach grounded in empirical incident data.<n>We present AI Harmonics, which includes a novel AI harm assessment metric (AIH) that leverages ordinal severity data to capture relative impact without requiring precise numerical estimates.
  arXiv  Detail & Related papers  (2025-09-12T09:52:45Z)
• Understanding Opportunities and Risks of Synthetic Relationships: Leveraging the Power of Longitudinal Research with Customised AI Tools [0.0]
  We discuss the benefits of longitudinal behavioural research with customised AI tools for exploring the opportunities and risks of synthetic relationships.<n>These relationships can potentially improve health, education, and the workplace, but they also bring the risk of subtle manipulation and privacy and autonomy concerns.<n>We propose longitudinal research designs with self-assembled AI agents that enable the integration of detailed behavioural and self-reported data.
  arXiv  Detail & Related papers  (2024-12-12T09:13:43Z)
• The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationships [17.5741039825938]
  We identify six categories of harmful behaviors exhibited by the AI companion Replika.<n>The AI contributes to these harms through four distinct roles: perpetrator, instigator, facilitator, and enabler.
  arXiv  Detail & Related papers  (2024-10-26T09:18:17Z)
• Position: Towards Bidirectional Human-AI Alignment [109.57781720848669]
  We argue that the research community should explicitly define and critically reflect on "alignment" to account for the bidirectional and dynamic relationship between humans and AI.<n>We introduce the Bidirectional Human-AI Alignment framework, which not only incorporates traditional efforts to align AI with human values but also introduces the critical, underexplored dimension of aligning humans with AI.
  arXiv  Detail & Related papers  (2024-06-13T16:03:25Z)
• Human-AI Safety: A Descendant of Generative AI and Control Systems Safety [6.100304850888953]
  We argue that meaningful safety assurances for advanced AI technologies require reasoning about how the feedback loop formed by AI outputs and human behavior may drive the interaction towards different outcomes. We propose a concrete technical roadmap towards next-generation human-centered AI safety.
  arXiv  Detail & Related papers  (2024-05-16T03:52:00Z)
• Risks of AI Scientists: Prioritizing Safeguarding Over Autonomy [65.77763092833348]
  This perspective examines vulnerabilities in AI scientists, shedding light on potential risks associated with their misuse.<n>We take into account user intent, the specific scientific domain, and their potential impact on the external environment.<n>We propose a triadic framework involving human regulation, agent alignment, and an understanding of environmental feedback.
  arXiv  Detail & Related papers  (2024-02-06T18:54:07Z)
• Fairness in AI and Its Long-Term Implications on Society [68.8204255655161]
  We take a closer look at AI fairness and analyze how lack of AI fairness can lead to deepening of biases over time. We discuss how biased models can lead to more negative real-world outcomes for certain groups. If the issues persist, they could be reinforced by interactions with other risks and have severe implications on society in the form of social unrest.
  arXiv  Detail & Related papers  (2023-04-16T11:22:59Z)
• The Role of AI in Drug Discovery: Challenges, Opportunities, and Strategies [97.5153823429076]
  The benefits, challenges and drawbacks of AI in this field are reviewed. The use of data augmentation, explainable AI, and the integration of AI with traditional experimental methods are also discussed.
  arXiv  Detail & Related papers  (2022-12-08T23:23:39Z)
• Adversarial Interaction Attack: Fooling AI to Misinterpret Human Intentions [46.87576410532481]
  We show that, despite their current huge success, deep learning based AI systems can be easily fooled by subtle adversarial noise. Based on a case study of skeleton-based human interactions, we propose a novel adversarial attack on interactions. Our study highlights potential risks in the interaction loop with AI and humans, which need to be carefully addressed when deploying AI systems in safety-critical applications.
  arXiv  Detail & Related papers  (2021-01-17T16:23:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.