GameWAM: A World Action Model for Video Games

• URL: http://arxiv.org/abs/2608.26200v1
• Date: Tue, 25 Aug 2026 17:21:15 GMT
• Title: GameWAM: A World Action Model for Video Games
• Abstract summary: We introduce GameWAM, to our knowledge the first WAM for native closed-loop gameplay and GUI control.<n>To support joint world-action learning, we construct synchronized gameplay and GUI trajectories.<n>Experiments demonstrate competitive task success with fewer executed native actions than the compared agents.
• Score: 36.43342575951464
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern video games combine first-person perception, rapid visual changes, persistent world state, and heterogeneous native controls. Existing game agents map visual and task context directly to actions but lack explicit world dynamics modeling, whereas interactive game world models predict visual futures from supplied actions but do not serve as task policies. World-Action Models (WAMs) unify these objectives, but remain largely unexplored under the dynamics and open-ended interaction of video games. We introduce GameWAM, to our knowledge the first WAM for native closed-loop gameplay and GUI control. GameWAM jointly generates future visual observations and executable keyboard-mouse trajectories through parallel visual and action generative processes with block-causal conditioning and flow matching. To support joint world-action learning, we construct synchronized gameplay and GUI trajectories. To handle heterogeneous native control, GameWAM predicts a gameplay/GUI mode at each action step and generates actions with mode-specific prediction distributions and continuous-action normalization. For long-horizon interaction, block-cycle control predicts beyond the committed horizon, executes only a short action prefix, and replans from new observations, while fine-grained within-cycle context and hierarchical cross-cycle history preserve temporal continuity. Experiments demonstrate competitive task success with fewer executed native actions than the compared agents. We further uncover Low-Frequency Action Source Imprinting (LASI), in which low-frequency components of the sampled action source systematically steer coarse generated camera motion under fixed conditioning, revealing a source-sensitivity failure mode in generative control. Project page is available at https://yunncheng.github.io/GameWAM/.

Related papers

• Valerant: An Automatic Navigable Game Map Generator via Action-Conditioned World Model Exploration [10.110436795631793]
  World Action Models (WAMs) couple predictive world modeling with action generation, allowing anticipated future states to guide agent behavior.<n>We present textscValerant, a framework that transforms a pretrained action-conditioned world model into a WAM for exploring and constructing 3D game maps.<n>This framework extends WAM-based interaction beyond 2D visual simulation and offers a new approach to reducing manual effort in 3D game-map creation.
  arXiv  Detail & Related papers  (2026-09-08T20:17:48Z)
• From Pixels to States: Rethinking Interactive World Models as Game Engines [32.66997200688335]
  This paper examines interactive game world modeling along four dimensions: player action control, game state dynamics, state-observation persistence, and real-time interactive generation.<n>We present a scalable data engine for Black Myth: Wukong that collects over 90 hours of gameplay with frame-aligned player actions, ground-truth game states, and visual observations.
  arXiv  Detail & Related papers  (2026-07-15T17:48:48Z)
• From Imagined Futures to Executable Actions: Mixture of Latent Actions for Robot Manipulation [88.39072412680633]
  We propose MoLA, a control-oriented interface that transforms imagined future videos into executable representations.<n>We evaluate our approach on simulated benchmarks and real-world robot manipulation tasks.
  arXiv  Detail & Related papers  (2026-05-12T14:15:16Z)
• GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents [76.60994803070436]
  GameWorld is a benchmark for evaluation of Multimodal Large Language Model (MLLM) game agents in browser environments.<n>Two game agent interfaces are studied: (i) computer-use agents that directly emit keyboard and mouse controls, and (ii) generalist multimodal agents that act in a semantic action space.<n>Results across 18 model-interface pairs suggest that even the best performing agent is far from achieving human capabilities on video games.
  arXiv  Detail & Related papers  (2026-04-08T17:49:03Z)
• ActionParty: Multi-Subject Action Binding in Generative Video Games [117.52562594944679]
  ActionParty is a controllable multi-subject world model for generative video games.<n>We demonstrate the first video world model capable of controlling up to seven players simultaneously across 46 diverse environments.
  arXiv  Detail & Related papers  (2026-04-02T17:59:58Z)
• Olaf-World: Orienting Latent Actions for Video World Modeling [100.96069208914957]
  Scaling action-controllable world models is limited by the scarcity of action labels.<n>We present Olaf-World, a pipeline that pretrains action-conditioned video world models from large-scale passive video.
  arXiv  Detail & Related papers  (2026-02-10T18:58:41Z)
• Astra: General Interactive World Model with Autoregressive Denoising [73.6594791733982]
  Astra is an interactive general world model that generates real-world futures for diverse scenarios.<n>We propose an autoregressive denoising architecture and use temporal causal attention to aggregate past observations.<n>Astra achieves interactive, consistent, and general long-term video prediction and supports various forms of interactions.
  arXiv  Detail & Related papers  (2025-12-09T18:59:57Z)
• Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model [19.937724706042804]
  Hunyuan-GameCraft-2 is a new paradigm of instruction-driven interaction for generative game world modeling.<n>Our model allows users to control game video contents through natural language prompts, keyboard, or mouse signals.<n>Our model generates temporally coherent and causally grounded interactive game videos.
  arXiv  Detail & Related papers  (2025-11-28T18:26:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.