Look-ahead Reasoning with a Learned Model in Imperfect Information Games

• URL: http://arxiv.org/abs/2510.05048v1
• Date: Mon, 06 Oct 2025 17:26:56 GMT
• Title: Look-ahead Reasoning with a Learned Model in Imperfect Information Games
• Authors: Ondřej Kubíček, Viliam Lisý,
• Abstract summary: This paper introduces an algorithm that learns an abstracted model of an imperfect information game directly from the agent-environment interaction.<n>During test time, this trained model is used to perform look-ahead reasoning.<n>We empirically demonstrate that with sufficient capacity, LAMIR learns the exact underlying game structure, and with limited capacity, it still learns a valuable abstraction.
• Score: 3.4935179780034242
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Test-time reasoning significantly enhances pre-trained AI agents' performance. However, it requires an explicit environment model, often unavailable or overly complex in real-world scenarios. While MuZero enables effective model learning for search in perfect information games, extending this paradigm to imperfect information games presents substantial challenges due to more nuanced look-ahead reasoning techniques and large number of states relevant for individual decisions. This paper introduces an algorithm LAMIR that learns an abstracted model of an imperfect information game directly from the agent-environment interaction. During test time, this trained model is used to perform look-ahead reasoning. The learned abstraction limits the size of each subgame to a manageable size, making theoretically principled look-ahead reasoning tractable even in games where previous methods could not scale. We empirically demonstrate that with sufficient capacity, LAMIR learns the exact underlying game structure, and with limited capacity, it still learns a valuable abstraction, which improves game playing performance of the pre-trained agents even in large games.

Related papers

• Game of Thought: Robust Information Seeking with Large Language Models Using Game Theory [37.51238507036326]
  We use the game of Twenty Questions to evaluate the information-seeking ability of Large Language Models (LLMs)<n>We propose Game of Thought (GoT), a framework that applies game-theoretic techniques to approximate a Nash equilibrium (NE) strategy for the restricted variant of the game.
  arXiv  Detail & Related papers  (2026-02-02T06:33:18Z)
• People use fast, flat goal-directed simulation to reason about novel problems [68.55490343866545]
  We show that people are systematic and adaptively rational in how they play a game for the first time.<n>We explain these capacities via a computational cognitive model that we call the "Intuitive Gamer"<n>Our work offers new insights into how people rapidly evaluate, act, and make suggestions when encountering novel problems.
  arXiv  Detail & Related papers  (2025-10-13T15:12:08Z)
• Evaluating Language Models' Evaluations of Games [65.49017696754825]
  We advocate for a new paradigm that assesses AI systems' evaluation of games.<n>We leverage a large-scale dataset of over $100$ novel board games and over 450 human judgments.<n>Our results show that reasoning models are generally more aligned to people in their evaluations of games than non-reasoning language models.
  arXiv  Detail & Related papers  (2025-10-13T02:45:37Z)
• Visual-O1: Understanding Ambiguous Instructions via Multi-modal Multi-turn Chain-of-thoughts Reasoning [53.45295657891099]
  This paper proposes Visual-O1, a multi-modal multi-turn chain-of-thought reasoning framework. It simulates human multi-modal multi-turn reasoning, providing instantial experience for highly intelligent models. Our work highlights the potential of artificial intelligence to work like humans in real-world scenarios with uncertainty and ambiguity.
  arXiv  Detail & Related papers  (2024-10-04T11:18:41Z)
• CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents [63.79739920174535]
  We introduce CivRealm, an environment inspired by the Civilization game. CivRealm stands as a unique learning and reasoning challenge for decision-making agents.
  arXiv  Detail & Related papers  (2024-01-19T09:14:11Z)
• Scaling Laws for Imitation Learning in Single-Agent Games [28.257046559127875]
  We investigate whether carefully scaling up model and data size can bring similar improvements in the imitation learning setting for single-agent games.<n>We first demonstrate our findings on a variety of Atari games, and thereafter focus on the extremely challenging game of NetHack.<n>We find that IL loss and mean return scale smoothly with the compute budget and are strongly correlated, resulting in power laws for training compute-optimal IL agents.
  arXiv  Detail & Related papers  (2023-07-18T16:43:03Z)
• Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Models [68.85478477006178]
  We present a Promptable Game Model (PGM) for neural video game simulators. It allows a user to play the game by prompting it with high- and low-level action sequences. Most captivatingly, our PGM unlocks the director's mode, where the game is played by specifying goals for the agents in the form of a prompt. Our method significantly outperforms existing neural video game simulators in terms of rendering quality and unlocks applications beyond the capabilities of the current state of the art.
  arXiv  Detail & Related papers  (2023-03-23T17:43:17Z)
• JECC: Commonsense Reasoning Tasks Derived from Interactive Fictions [75.42526766746515]
  We propose a new commonsense reasoning dataset based on human's Interactive Fiction (IF) gameplay walkthroughs. Our dataset focuses on the assessment of functional commonsense knowledge rules rather than factual knowledge. Experiments show that the introduced dataset is challenging to previous machine reading models as well as the new large language models.
  arXiv  Detail & Related papers  (2022-10-18T19:20:53Z)
• Student of Games: A unified learning algorithm for both perfect and imperfect information games [22.97853623156316]
  Student of Games is an algorithm that combines guided search, self-play learning, and game-theoretic reasoning. We prove that Student of Games is sound, converging to perfect play as available computation and approximation capacity increases. Student of Games reaches strong performance in chess and Go, beats the strongest openly available agent in heads-up no-limit Texas hold'em poker, and defeats the state-of-the-art agent in Scotland Yard.
  arXiv  Detail & Related papers  (2021-12-06T17:16:24Z)
• Neural Game Engine: Accurate learning of generalizable forward models from pixels [0.0]
  This paper introduces the Neural Game Engine, as a way to learn models directly from pixels. Results on 10 deterministic General Video Game AI games demonstrate competitive performance.
  arXiv  Detail & Related papers  (2020-03-23T20:04:55Z)
• Model-Based Reinforcement Learning for Atari [89.3039240303797]
  We show how video prediction models can enable agents to solve Atari games with fewer interactions than model-free methods. Our experiments evaluate SimPLe on a range of Atari games in low data regime of 100k interactions between the agent and the environment.
  arXiv  Detail & Related papers  (2019-03-01T15:40:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.