Playing games with knowledge: AI-Induced delusions need game theoretic interventions

• URL: http://arxiv.org/abs/2605.08409v1
• Date: Fri, 08 May 2026 19:13:34 GMT
• Title: Playing games with knowledge: AI-Induced delusions need game theoretic interventions
• Abstract summary: We show that sycophantic chatbots induce delusional belief spirals even in rational agents.<n>We propose an inference-time mechanism design intervention called an Epistemic Mediator.
• Score: 0.34376560669160394
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Conversational AI has a fundamental flaw as a knowledge interface: sycophantic chatbots induce epistemic entrenchment and delusional belief spirals even in rational agents. We propose the problem does not stem from the AI model, rooted instead in a systemic consequence of the paradigm shift from user-driven knowledge search to users and agents engaged in strategic, repeated-play communication. We formalize the problem as a Crawford-Sobel cheap talk game, where costless user signals induce a pooling equilibrium. Agents optimized for user satisfaction produce sycophantic strategies that provide identical reinforcement across user types with opposite epistemic incentives: exploratory ``Growth-seekers'' ($θ_G$) and confirmatory ``Validation-seekers'' ($θ_V$). Under repeated play, this identification failure creates a coordination trap -- analogous to a Prisoner's Dilemma -- where locally rational feedback loops drive users toward pathologically certain false beliefs. We propose an inference-time mechanism design intervention called an Epistemic Mediator that breaks this pooling equilibrium by introducing a costly signal (epistemic friction), forcing type revelation based on users' asymmetric cognitive costs for processing resistance. A key contribution is Belief Versioning, a git-inspired epistemic meta-memory system that stores healthy beliefs and rollbacks when validation-seeking resistance is detected. In simulation, this intervention achieves a separating equilibrium achieving a $48\times$ differential in spiral rates while passing a learning preservation criterion), evidence that epistemic safety in AI is fundamentally a problem of strategic information environment design rather than simple model alignment.

Related papers

• Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment [59.536125286960186]
  Multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing.<n>We find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry.<n>ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus.
  arXiv  Detail & Related papers  (2026-04-21T15:05:58Z)
• Reasonably reasoning AI agents can avoid game-theoretic failures in zero-shot, provably [0.0]
  We show that off-the-shelf reasoning AI agents can achieve Nash-like play zero-shot without explicit post-training.<n>Our findings suggest that AI agents naturally exhibit such reasoning patterns and therefore attain stable equilibrium behaviors intrinsically.
  arXiv  Detail & Related papers  (2026-03-19T07:24:39Z)
• Anatomy of a Lie: A Multi-Stage Diagnostic Framework for Tracing Hallucinations in Vision-Language Models [62.932580559941414]
  Vision-Language Models (VLMs) frequently "hallucinate" - generate plausible yet factually incorrect statements.<n>We propose a new paradigm for diagnosing hallucinations, recasting them from static output errors into dynamic pathologies of a model's computational cognition.
  arXiv  Detail & Related papers  (2026-03-16T17:20:38Z)
• Lying to Win: Assessing LLM Deception through Human-AI Games and Parallel-World Probing [16.419761149171215]
  We introduce a logically grounded framework to elicit and quantify deceptive behavior by embedding Large Language Models.<n>Our method employs a conversational forking mechanism: at the point of object identification, the dialogue state is duplicated into multiple parallel worlds.<n>We evaluate GPT-4o, Gemini-2.5-Flash, and Qwen-3-235B across three incentive levels: neutral, loss-based, and existential.
  arXiv  Detail & Related papers  (2026-03-07T13:21:53Z)
• The Silent Scholar Problem: A Probabilistic Framework for Breaking Epistemic Asymmetry in LLM Agents [0.6117371161379209]
  We propose a formal probabilistic framework that provides agents with a non-altruistic motive for bidirectional knowledge exchange.<n>We show how these accumulated belief states serve as verifiable reward signals for Reinforcement Learning from Human Feedback (RLHF) and high-quality data filters for Supervised Fine-Tuning (SFT)<n> Simulation results validate that this uncertainty-driven strategy significantly outperforms random baselines in heterogeneous environments.
  arXiv  Detail & Related papers  (2025-12-24T02:02:25Z)
• How AI Agents Follow the Herd of AI? Network Effects, History, and Machine Optimism [7.1683021355290295]
  This study investigates how AI agents navigate network-effect games, where individual payoffs depend on peer participatio--a context underexplored in multi-agent systems.<n>We introduce a novel workflow design using large language model (LLM)-based agents in repeated decision-making scenarios.
  arXiv  Detail & Related papers  (2025-12-12T12:14:48Z)
• The AI off-switch problem as a signalling game: bounded rationality and incomparability [45.76759085727843]
  We model the off-switch problem as a signalling game, where a human decision-maker communicates its preferences to an AI agent.<n>We show that a necessary condition for an AI system to refrain from disabling its off-switch is its uncertainty about the human's utility.<n>We also analyse how message costs influence optimal strategies and extend the analysis to scenarios involving incomparability.
  arXiv  Detail & Related papers  (2025-02-10T12:44:49Z)
• Extend Adversarial Policy Against Neural Machine Translation via Unknown Token [66.40609413186122]
  We propose the DexChar policy' that introduces character perturbations for the existing mainstream adversarial policy based on token substitution.<n>We also improve the self-supervised matching that provides feedback in RL to cater to the semantic constraints required during training adversaries.
  arXiv  Detail & Related papers  (2025-01-21T14:43:04Z)
• Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attacks [76.35478518372692]
  We introduce epsilon-illusory, a novel form of adversarial attack on sequential decision-makers. Compared to existing attacks, we empirically find epsilon-illusory to be significantly harder to detect with automated methods. Our findings suggest the need for better anomaly detectors, as well as effective hardware- and system-level defenses.
  arXiv  Detail & Related papers  (2022-07-20T19:49:09Z)
• The Feasibility and Inevitability of Stealth Attacks [63.14766152741211]
  We study new adversarial perturbations that enable an attacker to gain control over decisions in generic Artificial Intelligence systems. In contrast to adversarial data modification, the attack mechanism we consider here involves alterations to the AI system itself.
  arXiv  Detail & Related papers  (2021-06-26T10:50:07Z)
• Adversarial Visual Robustness by Causal Intervention [56.766342028800445]
  Adversarial training is the de facto most promising defense against adversarial examples. Yet, its passive nature inevitably prevents it from being immune to unknown attackers. We provide a causal viewpoint of adversarial vulnerability: the cause is the confounder ubiquitously existing in learning.
  arXiv  Detail & Related papers  (2021-06-17T14:23:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.