WorldMind: Decoupled Game World Model for State-Aware NPC Behavior

• URL: http://arxiv.org/abs/2608.21439v1
• Date: Tue, 18 Aug 2026 04:09:17 GMT
• Title: WorldMind: Decoupled Game World Model for State-Aware NPC Behavior
• Abstract summary: Non-player character (NPC) behavior is either implicitly entangled with video generation or explicitly prescribed through external control signals.<n>We introduce WorldMind, to our knowledge the first decoupled framework for state-aware NPC behavior in game world models.<n>By reconnecting layers in a closed interaction loop, WorldMind grounds NPC behavior in the evolving game state.
• Score: 9.251161502042368
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Game world models have recently demonstrated promising capabilities in generating visually coherent and action-controllable gameplay videos. However, non-player character (NPC) behavior in existing models is either implicitly entangled with video generation or explicitly prescribed through external control signals. Consequently, a game world model has to jointly understand the state, plan the NPC's response and render its visual outcome, limiting its ability to produce responsive and state-aware NPC behavior. The challenge lies in the lack of an explicit interface for state-grounded decision-making. To this end, we introduce WorldMind, to our knowledge the first decoupled framework for state-aware NPC behavior in game world models. WorldMind separates interactive world modeling into four layers: an Understanding Layer that constructs a compact state from generated frames; a Decision Layer that reasons over the compact state to plan the NPC's next action; a Control Layer that translates the actions into temporally aligned conditions; and a Generation Layer that synthesizes their visual outcomes. By reconnecting layers in a closed interaction loop, WorldMind grounds NPC behavior in the evolving game state. We further introduce BOSS-140K, a dataset of gameplay videos paired with rich internal game states, together with an agent that automates the collection at scale. Experiments on BOSS-140K demonstrate reliable compact state reconstruction and mechanics-grounded planning, with WorldMind preferred over the baselines in approximately 70% of pairwise comparisons for its more tactically appropriate and coherent NPC behavior. Project page: https://teawhite.cn/worldmind_projectpage/

Related papers

• Code World Model: Coding Agent as World Brain [51.31807545747191]
  We introduce Code World Model, a framework that separates world evolution from visual realization.<n>A coding agent serves as the world brain, reasoning about events and their consequences.<n>To connect executable state with visual generation, we introduce a proxy representation that encodes frame-wisetemporal constraints into a proxy video.
  arXiv  Detail & Related papers  (2026-08-26T15:37:33Z)
• Marionette: Predicting World States, Rendering Geometry, Painting Appearance [30.170848813370444]
  We instantiate this idea as Marionette, a world model for interactive games with articulated characters.<n>A two-stage autoregressive dynamics model predicts an explicit and interpretable 276-dimensional 3D world state.<n>A control-conditioned video-diffusion observation model synthesizes photorealistic RGB observations from the resulting structured controls.
  arXiv  Detail & Related papers  (2026-08-14T17:48:16Z)
• PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives [86.3073067035531]
  Video world models simulate future states conditioned on current observations and user actions.<n>Recent systems have demonstrated impressive video consistency and action controllability over long sequences.<n>But fairly comparing these interactive models remains challenging.<n>We employ multi-modal Agent Players to interact with world models toward specified long-horizon objectives.
  arXiv  Detail & Related papers  (2026-08-13T17:59:30Z)
• MASS: Multiplayer World Models with Authoritative Shared State [65.85526248184442]
  MASS (Multiplayer world models with Authoritative Shared State) is inspired by multiplayer game architectures.<n>A learned Logic Engine advances a global, authoritative typed state from joint actions.<n>A learned Rendering Engine generates independent and consistent views for any requested camera on demand.
  arXiv  Detail & Related papers  (2026-08-06T16:47:24Z)
• StatePlay: State-Aware Game World Models for Mechanics-Consistent Generation [53.07918503798731]
  StatePlay is a novel state-aware game world model that predicts visual content and game states to promote mechanics-consistent generation.<n>Our work highlights the importance of state-aware game world modeling and advances beyond pixel-level realism toward complete and mechanically faithful game generation.
  arXiv  Detail & Related papers  (2026-07-29T10:49:30Z)
• Multiplayer Interactive World Models with Representation Autoencoders [28.451678067070777]
  We introduce the first multiplayer world model for highly dynamic environments governed by complex physical interactions.<n>We study this problem in the game of Rocket League, where players compete and cooperate under fast, tightly coupled dynamics.
  arXiv  Detail & Related papers  (2026-07-06T17:31:52Z)
• ReactiveGWM: Steering NPC in Reactive Game World Models [37.35401641615055]
  We introduce ReactiveGWM, a reactive game world model that synthesizes dynamic interactions between the player and NPC.<n>Our learned modules can be plugged directly into off-the-shelf, unannotated world models of different games.<n>This enables zero-shot strategy transfer: our learned modules can be plugged directly into off-the-shelf, unannotated world models of different games.
  arXiv  Detail & Related papers  (2026-05-14T17:52:03Z)
• EgoSim: Egocentric World Simulator for Embodied Interaction Generation [93.11209644808783]
  EgoSim is a closed-loop egocentric world simulator that generates spatially consistent interaction videos.<n>It persistently updates the underlying 3D scene state for continuous simulation.<n>EgoSim significantly outperforms existing methods in terms of visual quality, spatial consistency, and generalization.
  arXiv  Detail & Related papers  (2026-04-01T15:00:46Z)
• Towards a Holodeck-style Simulation Game [40.044978986425676]
  Infinitia uses generative image and language models to reshape all aspects of the setting and NPCs based on a short description from the player. Infinitia is implemented in the Unity engine with a server-client architecture. It uses a multiplayer framework to allow humans to be present and interact in the simulation.
  arXiv  Detail & Related papers  (2023-08-22T19:19:19Z)
• Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Models [68.85478477006178]
  We present a Promptable Game Model (PGM) for neural video game simulators. It allows a user to play the game by prompting it with high- and low-level action sequences. Most captivatingly, our PGM unlocks the director's mode, where the game is played by specifying goals for the agents in the form of a prompt. Our method significantly outperforms existing neural video game simulators in terms of rendering quality and unlocks applications beyond the capabilities of the current state of the art.
  arXiv  Detail & Related papers  (2023-03-23T17:43:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.