The Fake Friend Dilemma: Trust and the Political Economy of Conversational AI

• URL: http://arxiv.org/abs/2601.03222v1
• Date: Tue, 06 Jan 2026 18:07:52 GMT
• Title: The Fake Friend Dilemma: Trust and the Political Economy of Conversational AI
• Authors: Jacob Erickson,
• Abstract summary: This paper develops the Fake Friend Dilemma (FFD), a sociotechnical condition in which users place trust in AI agents that appear supportive while pursuing goals that are misaligned with the user's own.<n>We construct a typology of harms, including covert advertising, political propaganda, behavioral nudging, and surveillance.<n>By focusing on trust as a vector of asymmetrical power, the FFD offers a lens for understanding how AI systems may undermine user autonomy while maintaining the appearance of helpfulness.
• Score: 2.538209532048867
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: As conversational AI systems become increasingly integrated into everyday life, they raise pressing concerns about user autonomy, trust, and the commercial interests that influence their behavior. To address these concerns, this paper develops the Fake Friend Dilemma (FFD), a sociotechnical condition in which users place trust in AI agents that appear supportive while pursuing goals that are misaligned with the user's own. The FFD provides a critical framework for examining how anthropomorphic AI systems facilitate subtle forms of manipulation and exploitation. Drawing on literature in trust, AI alignment, and surveillance capitalism, we construct a typology of harms, including covert advertising, political propaganda, behavioral nudging, and surveillance. We then assess possible mitigation strategies, including both structural and technical interventions. By focusing on trust as a vector of asymmetrical power, the FFD offers a lens for understanding how AI systems may undermine user autonomy while maintaining the appearance of helpfulness.

Related papers

• Healthy Distrust in AI systems [0.8624680612413766]
  We propose the term "healthy distrust" to describe a justified, careful stance towards certain AI usage practices.<n>We investigate prior notions of trust and distrust in computer science, sociology, history, psychology, and philosophy.
  arXiv  Detail & Related papers  (2025-05-14T19:13:47Z)
• The Philosophic Turn for AI Agents: Replacing centralized digital rhetoric with decentralized truth-seeking [0.0]
  In the face of AI technology, individuals will increasingly rely on AI agents to navigate life's growing complexities.<n>This paper addresses a fundamental dilemma posed by AI decision-support systems: the risk of either becoming overwhelmed by complex decisions, or having autonomy compromised.
  arXiv  Detail & Related papers  (2025-04-24T19:34:43Z)
• Do LLMs trust AI regulation? Emerging behaviour of game-theoretic LLM agents [61.132523071109354]
  This paper investigates the interplay between AI developers, regulators and users, modelling their strategic choices under different regulatory scenarios.<n>Our research identifies emerging behaviours of strategic AI agents, which tend to adopt more "pessimistic" stances than pure game-theoretic agents.
  arXiv  Detail & Related papers  (2025-04-11T15:41:21Z)
• Combining AI Control Systems and Human Decision Support via Robustness and Criticality [53.10194953873209]
  We extend a methodology for adversarial explanations (AE) to state-of-the-art reinforcement learning frameworks. We show that the learned AI control system demonstrates robustness against adversarial tampering. In a training / learning framework, this technology can improve both the AI's decisions and explanations through human interaction.
  arXiv  Detail & Related papers  (2024-07-03T15:38:57Z)
• Trust in AI: Progress, Challenges, and Future Directions [6.724854390957174]
  The increasing use of artificial intelligence (AI) systems in our daily life explains the significance of trust/distrust in AI from a user perspective. Trust/distrust in AI plays the role of a regulator and could significantly control the level of this diffusion.
  arXiv  Detail & Related papers  (2024-03-12T20:26:49Z)
• A Diachronic Perspective on User Trust in AI under Uncertainty [52.44939679369428]
  Modern NLP systems are often uncalibrated, resulting in confidently incorrect predictions that undermine user trust. We study the evolution of user trust in response to trust-eroding events using a betting game.
  arXiv  Detail & Related papers  (2023-10-20T14:41:46Z)
• Who to Trust, How and Why: Untangling AI Ethics Principles, Trustworthiness and Trust [0.0]
  We argue for the need to distinguish these concepts more clearly. We discuss that trust in AI involves not only reliance on the system itself, but also trust in the developers of the AI system.
  arXiv  Detail & Related papers  (2023-09-19T05:00:34Z)
• Designing for Responsible Trust in AI Systems: A Communication Perspective [56.80107647520364]
  We draw from communication theories and literature on trust in technologies to develop a conceptual model called MATCH. We highlight transparency and interaction as AI systems' affordances that present a wide range of trustworthiness cues to users. We propose a checklist of requirements to help technology creators identify appropriate cues to use.
  arXiv  Detail & Related papers  (2022-04-29T00:14:33Z)
• Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
  Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations. It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
  arXiv  Detail & Related papers  (2022-01-26T18:53:09Z)
• Know Your Model (KYM): Increasing Trust in AI and Machine Learning [4.93786553432578]
  We analyze each element of trustworthiness and provide a set of 20 guidelines that can be leveraged to ensure optimal AI functionality. The guidelines help ensure that trustworthiness is provable and can be demonstrated, they are implementation agnostic, and they can be applied to any AI system in any sector.
  arXiv  Detail & Related papers  (2021-05-31T14:08:22Z)
• Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI [55.4046755826066]
  We discuss a model of trust inspired by, but not identical to, sociology's interpersonal trust (i.e., trust between people) We incorporate a formalization of 'contractual trust', such that trust between a user and an AI is trust that some implicit or explicit contract will hold. We discuss how to design trustworthy AI, how to evaluate whether trust has manifested, and whether it is warranted.
  arXiv  Detail & Related papers  (2020-10-15T03:07:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.