Play Like Champions: Counterfactual Feedback Generation in Latent Space

• URL: http://arxiv.org/abs/2607.00190v1
• Date: Tue, 30 Jun 2026 21:15:34 GMT
• Title: Play Like Champions: Counterfactual Feedback Generation in Latent Space
• Abstract summary: We introduce Latent Maps of Performance, a framework for counterfactual path generation.<n>We trained a Guided Variational Autoencoder model on 23,305 professional tournament replays.<n>We conclude that there is a trade-off between the path-finding methods we employ and hope that future research will focus on developing model solutions for human improvement.
• Score: 10.090846922590462
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in reinforcement learning have produced superhuman agents across a wide range of competitive games. As a byproduct, researchers have begun studying how these agents play, extracting behavioral representations, analyzing decision structure, and modeling the latent geometry of expert performance. However, this growing body of work has overwhelmingly focused on defeating human players rather than providing feedback, leaving a critical gap in creating model solutions to improve human players. Unlike chess and Go, where AI has become integral to player training, real-time strategy (RTS) games lack principled frameworks for translating expert knowledge into actionable feedback. We introduce Latent Maps of Performance, a framework for counterfactual path generation. We focus on StarCraft~II data to model player improvement as an algorithmic recourse within a learned representation space. As inspiration for our work, we have looked at the championship model used in sports science. We trained a Guided Variational Autoencoder model on 23,305 professional tournament replays, enabling counterfactual traversal between losing and winning gameplay profiles. To fulfill our goal, we have devised and verified four traversal strategies on out-of-distribution (OOD) data randomly sampled from a dataset of amateur replays, namely linear interpolation, iterative optimal transport, density-regularized gradient ascent, and neural flow matching, each designed to generate multi-step improvement trajectories that remain grounded in observed expert behavior while moving a player's profile toward winning configurations. Feedback is extracted at multiple granularities to support players at different stages of improvement. Finally, we conclude that there is a trade-off between the path-finding methods we employ and hope that future research will focus on developing model solutions for human improvement.

Related papers

• Odysseus: Scaling VLMs to 100+ Turn Decision-Making in Games via Reinforcement Learning [50.464623632604976]
  In this work, we study RL-based training of vision-language models (VLMs) for long-horizon decision-making in Super Mario Land.<n>We propose an adapted variant of PPO with a lightweight turn-level critic, which substantially improves training stability and sample efficiency.<n>We introduce Odysseus, an open training framework for VLM agents, achieving substantial gains across multiple levels of the game.
  arXiv  Detail & Related papers  (2026-05-01T02:05:56Z)
• GenTac: Generative Modeling and Forecasting of Soccer Tactics [66.35532694378146]
  GenTac conceptualizes soccer tactics as a process over continuous multi-player trajectories and discrete semantic events.<n>It supports rich contextual conditioning, including opponent behavior, specific team or league playing styles, and strategic objectives.<n>It can be successfully trained to generalize to other dynamic team sports, including basketball, American football, and ice hockey.
  arXiv  Detail & Related papers  (2026-04-13T17:53:49Z)
• Game-TARS: Pretrained Foundation Models for Scalable Generalist Multimodal Game Agents [56.25101378553328]
  We present Game-TARS, a generalist game agent trained with a unified, scalable action space anchored to human-aligned keyboard-mouse inputs.<n>Game-TARS is pre-trained on over 500B tokens with diverse trajectories and multimodal data.<n> Experiments show that Game-TARS achieves about 2 times the success rate over the previous sota model on open-world Minecraft tasks.
  arXiv  Detail & Related papers  (2025-10-27T17:43:51Z)
• Human-Like Goalkeeping in a Realistic Football Simulation: a Sample-Efficient Reinforcement Learning Approach [35.515515697546554]
  This paper proposes a sample-efficient Deep Reinforcement Learning (DRL) method tailored for training and fine-tuning agents in industrial settings.<n>We evaluate our method training a goalkeeper agent in EA SPORTS FC 25, one of the best-selling football simulations today.<n>Our agent outperforms the game's built-in AI by 10% in ball saving rate.
  arXiv  Detail & Related papers  (2025-10-27T11:06:00Z)
• Play Style Identification Using Low-Level Representations of Play Traces in MicroRTS [0.0]
  Play style identification can provide valuable game design insights and enable adaptive experiences.<n>Previous work relies on domain knowledge to construct play trace representations using handcrafted features.<n>In this study, we explore the use of unsupervised CNN-LSTM autoencoder models to obtain latent representations directly from low-level play trace data.
  arXiv  Detail & Related papers  (2025-07-14T11:35:43Z)
• Behavioural Cloning in VizDoom [1.4999444543328293]
  This paper describes methods for training autonomous agents to play the game "Doom 2" through Imitation Learning (IL) We also explore how Reinforcement Learning (RL) compares to IL for humanness by comparing camera movement and trajectory data.
  arXiv  Detail & Related papers  (2024-01-08T16:15:43Z)
• Lucy-SKG: Learning to Play Rocket League Efficiently Using Deep Reinforcement Learning [0.0]
  We present Lucy-SKG, a Reinforcement Learning-based model that learned how to play Rocket League in a sample-efficient manner. Our contributions include the development of a reward analysis and visualization library, a novel parameterizable reward shape function, and auxiliary neural architectures.
  arXiv  Detail & Related papers  (2023-05-25T07:33:17Z)
• Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planning [27.593497502386143]
  Theory-Based Reinforcement Learning uses human-like intuitive theories to explore and model an environment. We instantiate the approach in a video game playing agent called EMPA. EMPA matches human learning efficiency on a suite of 90 Atari-style video games.
  arXiv  Detail & Related papers  (2021-07-27T01:38:13Z)
• Discovering Multi-Agent Auto-Curricula in Two-Player Zero-Sum Games [31.97631243571394]
  We introduce a framework, LMAC, that automates the discovery of the update rule without explicit human design. Surprisingly, even without human design, the discovered MARL algorithms achieve competitive or even better performance. We show that LMAC is able to generalise from small games to large games, for example training on Kuhn Poker and outperforming PSRO.
  arXiv  Detail & Related papers  (2021-06-04T22:30:25Z)
• Deep Policy Networks for NPC Behaviors that Adapt to Changing Design Parameters in Roguelike Games [137.86426963572214]
  Turn-based strategy games like Roguelikes, for example, present unique challenges to Deep Reinforcement Learning (DRL) We propose two network architectures to better handle complex categorical state spaces and to mitigate the need for retraining forced by design decisions.
  arXiv  Detail & Related papers  (2020-12-07T08:47:25Z)
• DeepCrawl: Deep Reinforcement Learning for Turn-based Strategy Games [137.86426963572214]
  We introduce DeepCrawl, a fully-playable Roguelike prototype for iOS and Android in which all agents are controlled by policy networks trained using Deep Reinforcement Learning (DRL) Our aim is to understand whether recent advances in DRL can be used to develop convincing behavioral models for non-player characters in videogames.
  arXiv  Detail & Related papers  (2020-12-03T13:53:29Z)
• Learning to Play Sequential Games versus Unknown Opponents [93.8672371143881]
  We consider a repeated sequential game between a learner, who plays first, and an opponent who responds to the chosen action. We propose a novel algorithm for the learner when playing against an adversarial sequence of opponents. Our results include algorithm's regret guarantees that depend on the regularity of the opponent's response.
  arXiv  Detail & Related papers  (2020-07-10T09:33:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.