Hierarchical Belief Modeling for Zero-Shot Opponent Adaptation in Partially Observable Multi-Agent Navigation

• URL: http://arxiv.org/abs/2609.12422v1
• Date: Fri, 11 Sep 2026 04:17:38 GMT
• Title: Hierarchical Belief Modeling for Zero-Shot Opponent Adaptation in Partially Observable Multi-Agent Navigation
• Abstract summary: Lux AI Season 3 requires agents to act under partial observability, randomized episode level dynamics, and a best of five match structure that rewards both tactical execution and fast adaptation.<n>We present HORIZON, a hierarchical agent that combines symmetry aware spatial perception, dual memory belief tracking, centric relic graph attention, information gain driven exploration, and an opponent conditioned policy mixture.
• Score: 9.529619955597216
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Lux AI Season 3 requires agents to act under partial observability, randomized episode level dynamics, and a best of five match structure that rewards both tactical execution and fast adaptation. We present HORIZON, a hierarchical agent that combines symmetry aware spatial perception, dual memory belief tracking, relic centric graph attention, information gain driven exploration, and an opponent conditioned policy mixture. HORIZON separates short horizon control from cross match meta reasoning, while auxiliary belief and world model objectives stabilize learning. Trained with PPO in a large scale JAX simulator, the resulting agent explicitly infers hidden game parameters and opponent style. Experiments show consistent gains in match win rate, episode win rate, adaptation gain, and league rating over strong recurrent and feed forward baselines.

Related papers

• LLM-Guided Reinforcement Learning for Adaptive NPC Behavior in Multi-Agent Combat Games [0.0]
  We investigate a runtime strategy-selection framework in which a large language model (LLM) guides a trained RL policy without modifying its underlying behavior.<n>We evaluate both configurations against three scripted opponent types across 600 episodes and analyze outcomes using the Mann-Whitney U test.
  arXiv  Detail & Related papers  (2026-08-27T07:17:58Z)
• Towards Improving Sequential Decision-Making in LLM Agents via Experience Memory [3.4935179780034242]
  Large language models have improved substantially on single-shot reasoning tasks, but their performance in sequential decision-making is less well understood.<n>We study this on fully-observable two-player zero-sum games, which provide ground-truth evaluation.<n>We show that post-game reflection and rule extraction yield measurable improvements on tic-tac-toe without modifying the model weights.
  arXiv  Detail & Related papers  (2026-08-04T10:12:12Z)
• Spatial Reasoning in LLM Game Agents: Impact of Causal Context and Multi-Step Planning [56.67877550139192]
  We conduct experiments across varying model scales, reasoning modes, and planning horizons.<n>We introduce a focused GVGAI benchmark consisting of three custom games with five difficulty levels to isolate spatial navigation.<n>Our results show that while larger models with an enabled thinking mode identify their positions more accurately, overall performance in coordinate matching remains limited for smaller models.
  arXiv  Detail & Related papers  (2026-07-22T12:10:45Z)
• MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs [54.81359054218573]
  We introduce Mindgames, a multi-game arena and evaluation platform for large language models (LLMs)<n>Mindgames provides a unified interaction interface, TrueSkill-based rating, and full trajectory logging across four game environments.<n>We analyze 29,571 multi-agent games with turn-level observations, actions, and rewards, together with MG-Ref, a deterministic offline tournament protocol.
  arXiv  Detail & Related papers  (2026-05-28T07:33:47Z)
• GenTac: Generative Modeling and Forecasting of Soccer Tactics [66.35532694378146]
  GenTac conceptualizes soccer tactics as a process over continuous multi-player trajectories and discrete semantic events.<n>It supports rich contextual conditioning, including opponent behavior, specific team or league playing styles, and strategic objectives.<n>It can be successfully trained to generalize to other dynamic team sports, including basketball, American football, and ice hockey.
  arXiv  Detail & Related papers  (2026-04-13T17:53:49Z)
• Expandable Decision-Making States for Multi-Agent Deep Reinforcement Learning in Soccer Tactical Analysis [6.8055385768376615]
  Invasion team sports such as soccer produce a high-dimensional, strongly coupled state space as many players interact on a shared field.<n>Traditional rule-based analyses are intuitive, while modern predictive machine learning models often perform pattern-matching without explicit agent representations.<n>Here, we propose Expandable Decision-Making States (EDMS), a semantically enriched state representation that augments raw positions and velocities with relational variables.
  arXiv  Detail & Related papers  (2025-10-01T04:01:51Z)
• RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning [125.96848846966087]
  Training large language models (LLMs) as interactive agents presents unique challenges.<n>While reinforcement learning has enabled progress in static tasks, multi-turn agent RL training remains underexplored.<n>We propose StarPO, a general framework for trajectory-level agent RL, and introduce RAGEN, a modular system for training and evaluating LLM agents.
  arXiv  Detail & Related papers  (2025-04-24T17:57:08Z)
• Toward Optimal LLM Alignments Using Two-Player Games [86.39338084862324]
  In this paper, we investigate alignment through the lens of two-agent games, involving iterative interactions between an adversarial and a defensive agent. We theoretically demonstrate that this iterative reinforcement learning optimization converges to a Nash Equilibrium for the game induced by the agents. Experimental results in safety scenarios demonstrate that learning in such a competitive environment not only fully trains agents but also leads to policies with enhanced generalization capabilities for both adversarial and defensive agents.
  arXiv  Detail & Related papers  (2024-06-16T15:24:50Z)
• Data-Scarce Identification of Game Dynamics via Sum-of-Squares Optimization [29.568222003322344]
  We introduce the Side-Information Assisted Regression (SIAR) framework, designed to identify game dynamics in multiplayer normal-form games. SIAR is solved using sum-of-squares (SOS) optimization, resulting in a hierarchy of approximations that provably converge to the true dynamics of the system. We showcase that the SIAR framework accurately predicts player behavior across a spectrum of normal-form games, widely-known families of game dynamics, and strong benchmarks, even if the unknown system is chaotic.
  arXiv  Detail & Related papers  (2023-07-13T09:14:48Z)
• Finding mixed-strategy equilibria of continuous-action games without gradients using randomized policy networks [83.28949556413717]
  We study the problem of computing an approximate Nash equilibrium of continuous-action game without access to gradients. We model players' strategies using artificial neural networks. This paper is the first to solve general continuous-action games with unrestricted mixed strategies and without any gradient information.
  arXiv  Detail & Related papers  (2022-11-29T05:16:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.