The Yokai Learning Environment: Tracking Beliefs Over Space and Time

• URL: http://arxiv.org/abs/2508.12480v1
• Date: Sun, 17 Aug 2025 19:42:17 GMT
• Title: The Yokai Learning Environment: Tracking Beliefs Over Space and Time
• Authors: Constantin Ruhdorfer, Matteo Bortoletto, Andreas Bulling,
• Abstract summary: We introduce the Yokai Learning Environment (YLE) - a reinforcement learning environment based on the cooperative card game Yokai.<n>Success requires tracking evolving beliefs, remembering past observations, using hints as grounded communication, and maintaining common ground with teammates.<n>Current RL agents struggle to solve the YLE, even when given access to perfect memory.<n>While belief modelling improves performance, agents are still unable to effectively generalise to unseen partners or form accurate beliefs over longer games.
• Score: 8.882575080324711
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Developing collaborative AI hinges on Theory of Mind (ToM) - the ability to reason about the beliefs of others to build and maintain common ground. Existing ToM benchmarks, however, are restricted to passive observer settings or lack an assessment of how agents establish and maintain common ground over time. To address these gaps, we introduce the Yokai Learning Environment (YLE) - a multi-agent reinforcement learning (RL) environment based on the cooperative card game Yokai. In the YLE, agents take turns peeking at hidden cards and moving them to form clusters based on colour. Success requires tracking evolving beliefs, remembering past observations, using hints as grounded communication, and maintaining common ground with teammates. Our evaluation yields two key findings: First, current RL agents struggle to solve the YLE, even when given access to perfect memory. Second, while belief modelling improves performance, agents are still unable to effectively generalise to unseen partners or form accurate beliefs over longer games, exposing a reliance on brittle conventions rather than robust belief tracking. We use the YLE to investigate research questions in belief modelling, memory, partner generalisation, and scaling to higher-order ToM.

Related papers

• Beyond Words: Evaluating and Bridging Epistemic Divergence in User-Agent Interaction via Theory of Mind [8.740788873949471]
  Large Language Models (LLMs) have developed rapidly and are widely applied to both general-purpose and professional tasks.<n>They still struggle to comprehend and respond to the true user needs when intentions and instructions are imprecisely conveyed.
  arXiv  Detail & Related papers  (2026-02-14T16:01:59Z)
• Theory of Space: Can Foundation Models Construct Spatial Beliefs through Active Exploration? [83.13508919229939]
  Theory of Space is defined as an agent's ability to actively acquire information through self-directed, active exploration.<n>A key innovation is spatial belief probing, which prompts models to reveal their internal spatial representations at each step.<n>Our findings suggest that current foundation models struggle to maintain coherent, revisable spatial beliefs during active exploration.
  arXiv  Detail & Related papers  (2026-02-04T19:06:40Z)
• LLM-Hanabi: Evaluating Multi-Agent Gameplays with Theory-of-Mind and Rationale Inference in Imperfect Information Collaboration Game [47.019077016616144]
  This study introduces LLM-Hanabi, a novel benchmark that uses the cooperative game Hanabi to evaluate the rationale inference and ToM.<n>Across a range of models, we find a significant positive correlation between ToM and in-game success.<n>We conclude that prioritizing first-order ToM is a promising direction for enhancing the collaborative capabilities of future models.
  arXiv  Detail & Related papers  (2025-10-06T16:17:24Z)
• Theory of Mind Using Active Inference: A Framework for Multi-Agent Cooperation [4.06613683722116]
  We present a novel approach to multi-agent cooperation by implementing theory of mind (ToM) within active inference.<n>ToM enables agents to reason about others' beliefs while planning their own actions.<n>We extend the sophisticated inference tree-based planning algorithm to systematically explore joint policy spaces.
  arXiv  Detail & Related papers  (2025-08-01T08:02:35Z)
• Corrupted by Reasoning: Reasoning Language Models Become Free-Riders in Public Goods Games [87.5673042805229]
  How large language models balance self-interest and collective well-being is a critical challenge for ensuring alignment, robustness, and safe deployment.<n>We adapt a public goods game with institutional choice from behavioral economics, allowing us to observe how different LLMs navigate social dilemmas.<n>Surprisingly, we find that reasoning LLMs, such as the o1 series, struggle significantly with cooperation.
  arXiv  Detail & Related papers  (2025-06-29T15:02:47Z)
• The Decrypto Benchmark for Multi-Agent Reasoning and Theory of Mind [8.341160422849969]
  Decrypto is a game-based benchmark for multi-agent reasoning and ToM.<n>It is the first platform for designing interactive ToM experiments.<n>We find that LLM game-playing abilities lag behind humans and simple word-embedding baselines.
  arXiv  Detail & Related papers  (2025-06-25T17:55:27Z)
• Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning [87.7836502955847]
  We propose a novel self-rewarding reinforcement learning framework to enhance Large Language Model (LLM) reasoning.<n>Our key insight is that correct responses often exhibit consistent trajectory patterns in terms of model likelihood.<n>We introduce CoVo, an intrinsic reward mechanism that integrates Consistency and Volatility via a robust vector-space aggregation strategy.
  arXiv  Detail & Related papers  (2025-06-10T12:40:39Z)
• The Traitors: Deception and Trust in Multi-Agent Language Model Simulations [0.0]
  We introduce The Traitors, a multi-agent simulation framework inspired by social deduction games.<n>We develop a suite of evaluation metrics capturing deception success, trust dynamics, and collective inference quality.<n>Our initial experiments across DeepSeek-V3, GPT-4o-mini, and GPT-4o (10 runs per model) reveal a notable asymmetry.
  arXiv  Detail & Related papers  (2025-05-19T10:01:35Z)
• ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learning [53.817538122688944]
  We introduce Reinforced Meta-thinking Agents (ReMA) to elicit meta-thinking behaviors from Reasoning of Large Language Models (LLMs)<n>ReMA decouples the reasoning process into two hierarchical agents: a high-level meta-thinking agent responsible for generating strategic oversight and plans, and a low-level reasoning agent for detailed executions.<n> Empirical results from single-turn experiments demonstrate that ReMA outperforms single-agent RL baselines on complex reasoning tasks.
  arXiv  Detail & Related papers  (2025-03-12T16:05:31Z)
• Evaluating and Enhancing LLMs Agent based on Theory of Mind in Guandan: A Multi-Player Cooperative Game under Imperfect Information [36.11862095329315]
  Large language models (LLMs) have shown success in handling simple games with imperfect information. This study investigates the applicability of knowledge acquired by open-source and API-based LLMs to sophisticated text-based games.
  arXiv  Detail & Related papers  (2024-08-05T15:36:46Z)
• Toward Optimal LLM Alignments Using Two-Player Games [86.39338084862324]
  In this paper, we investigate alignment through the lens of two-agent games, involving iterative interactions between an adversarial and a defensive agent. We theoretically demonstrate that this iterative reinforcement learning optimization converges to a Nash Equilibrium for the game induced by the agents. Experimental results in safety scenarios demonstrate that learning in such a competitive environment not only fully trains agents but also leads to policies with enhanced generalization capabilities for both adversarial and defensive agents.
  arXiv  Detail & Related papers  (2024-06-16T15:24:50Z)
• Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation [52.930183136111864]
  We propose using scorable negotiation to evaluate Large Language Models (LLMs) To reach an agreement, agents must have strong arithmetic, inference, exploration, and planning capabilities. We provide procedures to create new games and increase games' difficulty to have an evolving benchmark.
  arXiv  Detail & Related papers  (2023-09-29T13:33:06Z)
• Multiagent Inverse Reinforcement Learning via Theory of Mind Reasoning [0.0]
  We propose a novel approach to Multiagent Inverse Reinforcement Learning (MIRL) MIRL aims to infer the reward functions guiding the behavior of each individual given trajectories of a team's behavior during task performance. We evaluate our approach in a simulated 2-player search-and-rescue operation.
  arXiv  Detail & Related papers  (2023-02-20T19:07:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.