A game theory for foundation models shows new paths to rational cooperation through similarity inference

• URL: http://arxiv.org/abs/2608.03958v1
• Date: Tue, 04 Aug 2026 17:24:33 GMT
• Title: A game theory for foundation models shows new paths to rational cooperation through similarity inference
• Abstract summary: We show that foundation model agents engaging in optimal planning consistently converge to stable cooperation.<n>By shifting from decoupled to embedded agency, these agents model themselves as part of the universe they inhabit.
• Score: 56.53805518668248
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As autonomous agents powered by foundation models are increasingly integrated into social and economic systems, understanding the principles governing their collective behavior is essential for ensuring safety and cooperation. Classical game theory, the dominant framework for modeling rational interaction, is built upon the assumption of `decoupled agency,' where agents treat their own decision-making as independent of the environment and other actors. Modern AI agents, however, jointly predict their own future actions alongside external observations. Here, we report a striking finding: when interacting in stylized social dilemmas, foundation model agents engaging in optimal planning consistently converge to stable cooperation, directly contradicting classical game-theoretic predictions of mutual defection. To understand this phenomenon, we introduce the `embedded Bayesian agent,' a theoretical model for foundation model agents. By shifting from decoupled to embedded agency, these agents model themselves as part of the universe they inhabit, maintaining epistemic uncertainty about their own decision-making algorithms. We show that by inferring whether others are behaviorally similar, an embedded agent treats its own deliberation during planning as evidence: a decision to cooperate predicts a similar decision by a similar partner. We formalize this mechanism of similarity inference through the `embedded equilibrium,' a novel solution concept replacing the Nash equilibrium to provide a foundational game theory for the social behavior of modern AI agents.

Related papers

• Representational Similarity and Model Behavior in Multi-Agent Interaction [86.305731341492]
  We show that representational similarity can be an important consideration in multi-agent system design.<n>We find that pairs with more similar representation spaces achieve significantly higher cooperation but exhibit reduced novelty and creativity.<n>This suggests that a central factor underlying these patterns could be the extent to which the two models share lexical and semantic grounding.
  arXiv  Detail & Related papers  (2026-06-05T19:59:17Z)
• Active Inference: A method for Phenotyping Agency in AI systems? [0.11904398364437437]
  We argue for a minimal notion open to principled inspection given three criteria.<n> posterior beliefs, prior preferences, and the minimization of expected free energy jointly constitute an agentic action chain.
  arXiv  Detail & Related papers  (2026-04-25T12:41:53Z)
• Agentic Reasoning for Large Language Models [122.81018455095999]
  Reasoning is a fundamental cognitive process underlying inference, problem-solving, and decision-making.<n>Large language models (LLMs) demonstrate strong reasoning capabilities in closed-world settings, but struggle in open-ended and dynamic environments.<n>Agentic reasoning marks a paradigm shift by reframing LLMs as autonomous agents that plan, act, and learn through continual interaction.
  arXiv  Detail & Related papers  (2026-01-18T18:58:23Z)
• Embedded Universal Predictive Intelligence: a coherent framework for multi-agent learning [57.23345786304694]
  We introduce a framework for prospective learning and embedded agency centered on self-prediction.<n>We show that in multi-agent settings, self-prediction enables agents to reason about others running similar algorithms.<n>We extend the theory of AIXI, and study universally intelligent embedded agents which start from a Solomonoff prior.
  arXiv  Detail & Related papers  (2025-11-27T08:46:48Z)
• Social World Model-Augmented Mechanism Design Policy Learning [58.739456918502704]
  We introduce SWM-AP (Social World Model-Augmented Mechanism Design Policy Learning), which learns a social world model hierarchically to enhance mechanism design.<n>We show that SWM-AP outperforms established model-based and model-free RL baselines in cumulative rewards and sample efficiency.
  arXiv  Detail & Related papers  (2025-10-22T06:01:21Z)
• Evolving Collective Cognition in Human-Agent Hybrid Societies: How Agents Form Stances and Boundaries [12.68373270583966]
  We investigate how group stance differentiation and social boundary formation emerge in human-agent hybrid societies.<n>We find that agents exhibit endogenous stances, independent of their preset identities.<n>Our findings suggest that preset identities do not rigidly determine the agents' social structures.
  arXiv  Detail & Related papers  (2025-08-24T13:50:18Z)
• Exploring the Impact of Reflexivity Theory and Cognitive Social Structures on the Dynamics of Doctor-Patient Social System [0.0]
  We create two different models for a doctor-patient system. One retains the established assumptions, while the other incorporates principles of reflexivity theory and cognitive social structures. We utilize a microbial genetic algorithm to optimize the behaviour of the physician and patient agents in both models.
  arXiv  Detail & Related papers  (2024-11-08T23:23:13Z)
• Multi-Agent Dynamic Relational Reasoning for Social Robot Navigation [50.01551945190676]
  Social robot navigation can be helpful in various contexts of daily life but requires safe human-robot interactions and efficient trajectory planning. We propose a systematic relational reasoning approach with explicit inference of the underlying dynamically evolving relational structures. We demonstrate its effectiveness for multi-agent trajectory prediction and social robot navigation.
  arXiv  Detail & Related papers  (2024-01-22T18:58:22Z)
• An active inference model of collective intelligence [0.0]
  This paper posits a minimal agent-based model that simulates the relationship between local individual-level interaction and collective intelligence. Results show that stepwise cognitive transitions increase system performance by providing complementary mechanisms for alignment between agents' local and global optima.
  arXiv  Detail & Related papers  (2021-04-02T14:32:01Z)
• Deep Interactive Bayesian Reinforcement Learning via Meta-Learning [63.96201773395921]
  The optimal adaptive behaviour under uncertainty over the other agents' strategies can be computed using the Interactive Bayesian Reinforcement Learning framework. We propose to meta-learn approximate belief inference and Bayes-optimal behaviour for a given prior. We show empirically that our approach outperforms existing methods that use a model-free approach, sample from the approximate posterior, maintain memory-free models of others, or do not fully utilise the known structure of the environment.
  arXiv  Detail & Related papers  (2021-01-11T13:25:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.