Dreaming Of Others: Latent Teammate Modeling In World Models For Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2605.31361v1
• Date: Fri, 29 May 2026 14:34:50 GMT
• Title: Dreaming Of Others: Latent Teammate Modeling In World Models For Multi-Agent Reinforcement Learning
• Abstract summary: We introduce an architecture that factorizes the latent state of a Dreamer-style recurrent state-space model into environment and teammate components.<n>These teammate latents condition the actor and critic, enabling the agent to imagine and adapt to diverse collaborators.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In cooperative multi-agent reinforcement learning (MARL), agents must coordinate with partners whose internal policies and intentions are not directly observable. While world models such as Dreamer have demonstrated strong generalization and sample efficiency in single-agent settings, their application to MARL remains limited by an inability to handle teammate-induced uncertainty. We propose a new perspective: treat teammates as structured, learnable components within the agent's world model. We introduce an architecture that factorizes the latent state of a Dreamer-style recurrent state-space model (RSSM) into environment and teammate components, and learns an auxiliary Theory-of-Mind (ToM) head to infer latent embeddings of partner behavior such as character, intent, and predicted actions from partial trajectories. These teammate latents condition the actor and critic, enabling the agent to imagine and adapt to diverse collaborators. We outline how this approach can support zero-shot and few-shot coordination in partially observable settings and propose a set of benchmarks and evaluation protocols to assess its impact. This work positions world models as not only predictors of environmental dynamics, but as simulators of social behavior, opening new directions for generalizable, human-compatible AI.

Related papers

• From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence [54.62791372989525]
  Action models, vision-language--action policies, and world models have advanced this goal.<n>World Action Models (WAMs) are particularly promising because they connect candidate interventions with predicted consequences.<n>We review the evolution toward WAMs and organize these limitations into three coupled gaps: model roles and representations, objectives and standardization, and system composition.
  arXiv  Detail & Related papers  (2026-07-13T15:22:56Z)
• Humans' ALMANAC: A Human Collaboration Dataset of Action-Level Mental Model Annotations for Agent Collaboration [54.18136393879849]
  We present ALMANAC, a dataset of action-level mental model ANnotations for Agent Collaboration.<n>ALMANAC contains 2,987 collaboration actions, each paired with theory-informed mental model annotations.<n>Our results demonstrate ALMANAC's utility in evaluating models' ability to simulate human collaborative behaviors.
  arXiv  Detail & Related papers  (2026-06-04T16:56:12Z)
• SODE: Analyzing Social Dynamics in LLM Agents [17.885513252130373]
  We introduce SODE (Social Dynamics Evaluation), a framework that evaluates AI agents across three evolutionary dimensions.<n>Applying SODE reveals systematic divergences.<n>We demonstrate that a "long-horizon framing" can unlock reciprocal capabilities in reasoning models.
  arXiv  Detail & Related papers  (2026-05-06T14:50:07Z)
• Multi-Agent Model-Based Reinforcement Learning with Joint State-Action Learned Embeddings [10.36125908359289]
  We present a novel model-based multi-agent reinforcement learning framework.<n>We design a world model trained with variational auto-encoders and augment the model using the state-action learned embedding.<n>By coupling imagined trajectories with SALE-based action values, the agents acquire a richer understanding of how their choices influence collective outcomes.
  arXiv  Detail & Related papers  (2026-02-13T01:57:21Z)
• ReCollab: Retrieval-Augmented LLMs for Cooperative Ad-hoc Teammate Modeling [2.1130419553264015]
  Ad-hoc teamwork requires agents to infer the behavior of previously unseen teammates and adapt their policy accordingly.<n>We introduce Collab, a language-based framework that classifies partner types using a behavior derived from trajectory features.<n>We extend it to ReCollab, which incorporates retrieval-augmented generation (RAG) to stabilize inference with trajectories.
  arXiv  Detail & Related papers  (2025-12-05T12:33:11Z)
• Evaluating Generalization Capabilities of LLM-Based Agents in Mixed-Motive Scenarios Using Concordia [100.74015791021044]
  Large Language Model (LLM) agents have demonstrated impressive capabilities for social interaction.<n>Existing evaluation methods fail to measure how well these capabilities generalize to novel social situations.<n>We present empirical results from the NeurIPS 2024 Concordia Contest, where agents were evaluated on their ability to achieve mutual gains.
  arXiv  Detail & Related papers  (2025-12-03T00:11:05Z)
• Partner Modelling Emerges in Recurrent Agents (But Only When It Matters) [9.957208220679261]
  We train simple model-free RNN agents to collaborate with a population of diverse partners.<n>We find structured partner modelling emerges when agents can influence partner behaviour by controlling task allocation.<n>Our results show that partner modelling can arise spontaneously in model-free agents -- but only under environmental conditions that impose the right kind of social pressure.
  arXiv  Detail & Related papers  (2025-05-22T22:24:12Z)
• Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning [51.52387511006586]
  We propose Hierarchical Opponent modeling and Planning (HOP), a novel multi-agent decision-making algorithm. HOP is hierarchically composed of two modules: an opponent modeling module that infers others' goals and learns corresponding goal-conditioned policies. HOP exhibits superior few-shot adaptation capabilities when interacting with various unseen agents, and excels in self-play scenarios.
  arXiv  Detail & Related papers  (2024-06-12T08:48:06Z)
• COMBO: Compositional World Models for Embodied Multi-Agent Cooperation [65.46592503910875]
  We investigate the problem of embodied multi-agent cooperation, where decentralized agents must cooperate given only egocentric views of the world.<n>We learn a compositional world model for multi-agent cooperation by factorizing the naturally composable joint actions of multiple agents.<n>We evaluate our methods on three challenging benchmarks with 2-4 agents.
  arXiv  Detail & Related papers  (2024-04-16T17:59:11Z)
• Multi-Agent Imitation Learning with Copulas [102.27052968901894]
  Multi-agent imitation learning aims to train multiple agents to perform tasks from demonstrations by learning a mapping between observations and actions. In this paper, we propose to use copula, a powerful statistical tool for capturing dependence among random variables, to explicitly model the correlation and coordination in multi-agent systems. Our proposed model is able to separately learn marginals that capture the local behavioral patterns of each individual agent, as well as a copula function that solely and fully captures the dependence structure among agents.
  arXiv  Detail & Related papers  (2021-07-10T03:49:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.