Mind-Studio: Executable World Models with Lookahead Evaluation for Partially Observable Games

• URL: http://arxiv.org/abs/2606.16070v2
• Date: Tue, 16 Jun 2026 06:37:37 GMT
• Title: Mind-Studio: Executable World Models with Lookahead Evaluation for Partially Observable Games
• Abstract summary: Mind-Studio is a framework that synthesizes pygame-style world models from state-action-next-state trajectories.<n>On Montezuma's Revenge, Mind-Studio improves chosen-action next-state prediction from 0.3% for PoE-World to 48.7% while verifying 5 of 8 subgoals.
• Score: 26.359796001522707
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: World-model synthesis aims to turn interaction experience into an internal model of environment dynamics. Existing symbolic approaches often fit observed transitions or mixtures of local rules, but they do not produce a complete executable program that can run independently of the real environment. We present Mind-Studio, a framework that synthesizes executable pygame-style world models from state-action-next-state trajectories using large language models. Mind-Studio combines entropy-selected traces with a lightweight game skill file containing object, action, and static scene information extracted from screenshots. We evaluate synthesis quality with a K-step lookahead fidelity protocol that compares generated world-model rollouts against Real-ALE rollouts from the same state. On Montezuma's Revenge, Mind-Studio improves chosen-action next-state prediction from 0.3% for PoE-World to 48.7% while verifying 5 of 8 subgoals; across Alien, Assault, and Skiing, it achieves stronger branch-level fidelity than prior learned lookahead sources.

Related papers

• Programmable World Model [37.43654073749753]
  We introduce Programmable World Model, a framework that decouples world-state evolution from visual observation generation.<n>An agent translates natural-language instructions into executable programs that specify entity states and state-transition rules.<n>A lightweight engine executes these programs to update and maintain an explicit, persistent global world state.
  arXiv  Detail & Related papers  (2026-09-09T17:59:32Z)
• SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators [61.95179776027961]
  SyncWorld is an action-conditioned world model that serves as a zero-shot simulator across unseen environments without any additional training.<n>Training with visual calibration contexts teaches the model to interpret actions through visual evidence.<n>Experiments show that SyncWorld can accurately simulate action outcomes in previously unseen settings.
  arXiv  Detail & Related papers  (2026-09-08T17:59:47Z)
• KineBench: Benchmarking Embodied World Models via IDM-Free Kinematic Grounding [58.541098216700796]
  evaluating the physical consistency of embodied world models is a critical open challenge.<n>Existing frameworks almost exclusively rely on Inverse Dynamics Models(IDMs) for action extraction.<n>We present KineBench, an IDM-free closed-loop benchmark for EWMs.
  arXiv  Detail & Related papers  (2026-07-22T08:04:17Z)
• WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Models [37.59051701511746]
  We introduce WorldRoamBench, an open-world benchmark for long-horizon stability across four dimensions.<n>The benchmark comprises 600+ test cases across Nature, Urban, and Indoor scenes in first/third-person views with WASD 10-60s continuous interaction.
  arXiv  Detail & Related papers  (2026-06-30T13:51:44Z)
• PatchWorld: Gradient-Free Optimization of Executable World Models [69.05094454437402]
  We introduce PatchWorld, a gradient-free framework that turns offline trajectories into executable Python world models.<n>Across seven AgentGym environments, PatchWorld-Simple achieves the highest code-based planning score.<n>We find that a human-specified residual-memory bias improves surface observation fidelity but weakens decision utility.
  arXiv  Detail & Related papers  (2026-05-29T06:11:14Z)
• SceneCode: Executable World Programs for Editable Indoor Scenes with Articulated Objects [69.20984454755512]
  Indoor scene synthesis underpins embodied AI, robotic manipulation, and simulation-based policy evaluation.<n>Existing pipelines represent generated content as static meshes and inherit articulation only from curated asset libraries.<n>We present SceneCode, a framework that compiles a natural language prompt into an executable, code-driven indoor world.
  arXiv  Detail & Related papers  (2026-05-19T09:31:04Z)
• Ego2World: Compiling Egocentric Cooking Videos into Executable Worlds for Belief-State Planning [55.43343782036886]
  We introduce Ego2World, an executable benchmark that turns egocentric cooking videos into executable symbolic worlds governed by graph-transition rules.<n>During evaluation, the simulator maintains a hidden world graph, while the agent plans over its own partial belief graph using only local observations and execution feedback.<n>This separation forces agents to update memory and replan without observing the true world state.
  arXiv  Detail & Related papers  (2026-05-13T10:53:15Z)
• Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modeling [51.40150411616207]
  We introduce Latent Particle World Model (LPWM), a self-supervised object-centric world model scaled to real-world multi-object datasets.<n>LPWM autonomously discovers keypoints, bounding boxes, and object masks directly from video data.<n>Our architecture is trained end-to-end purely from videos and supports flexible conditioning on actions, language, and image goals.
  arXiv  Detail & Related papers  (2026-03-04T19:36:08Z)
• Infinite-World: Scaling Interactive World Models to 1000-Frame Horizons via Pose-Free Hierarchical Memory [101.2076718776139]
  We propose a robust interactive world model capable of maintaining coherent visual memory over 1000+ frames in complex real-world environments.<n>We introduce a Pose-free Memory (HPMC) that distills historical latents into a fixed-budget geometric representation.<n>We also propose an Uncertainty-aware Action Labeling module that discretizes continuous motion into a tri-state logic.
  arXiv  Detail & Related papers  (2026-02-02T17:52:56Z)
• A Data-Driven Discretized CS:GO Simulation Environment to Facilitate Strategic Multi-Agent Planning Research [1.1765015608581086]
  We present DECOY, a novel multi-agent simulator that abstracts strategic, long-horizon planning in 3D terrains into high-level discretized simulation.<n>Using Counter-Strike: Global Offensive as a testbed, our framework accurately simulates gameplay using only movement decisions as tactical positioning.
  arXiv  Detail & Related papers  (2025-09-08T06:02:59Z)
• COME: Adding Scene-Centric Forecasting Control to Occupancy World Model [18.815436110557112]
  World models are critical for autonomous driving to simulate environmental dynamics and generate synthetic data.<n>Existing methods struggle to disentangle ego-vehicle motion (perspective shifts from scene evolvement)<n>We propose to separate environmental changes from ego-motion by leveraging the scene-centric coordinate systems.
  arXiv  Detail & Related papers  (2025-06-16T09:01:09Z)
• MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraft [21.530000271719803]
  We propose MineWorld, a real-time interactive world model on Minecraft.<n>MineWorld is driven by a visual-action autoregressive Transformer, which takes paired game scenes and corresponding actions as input.<n>We develop a novel parallel decoding algorithm that predicts the spatial redundant tokens in each frame at the same time.
  arXiv  Detail & Related papers  (2025-04-11T09:41:04Z)
• Infusing Commonsense World Models with Graph Knowledge [89.27044249858332]
  We study the setting of generating narratives in an open world text adventure game. A graph representation of the underlying game state can be used to train models that consume and output both grounded graph representations and natural language descriptions and actions.
  arXiv  Detail & Related papers  (2023-01-13T19:58:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.