Distilling Game Code World Model Generation into Lightweight Large Language Models

• URL: http://arxiv.org/abs/2605.24375v1
• Date: Sat, 23 May 2026 03:30:36 GMT
• Title: Distilling Game Code World Model Generation into Lightweight Large Language Models
• Abstract summary: Large Language Models (LLMs) have shown great ability in generating executable code from natural language.<n>We study this problem in game settings, where generated environments must implement rules, legal actions, state transitions, observations, and rewards.<n>This work investigates whether Game Code World Models can be distilled into smaller models through post-training.
• Score: 8.471264611944559
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) have shown great ability in generating executable code from natural language, opening the possibility of automatically constructing environments for AI agents. Recent work on Code World Models (CWMs) demonstrates that LLMs can translate game rules into Python implementations compatible with solvers like Monte Carlo Tree Search. We study this problem in game settings, where generated environments must implement rules, legal actions, state transitions, observations, and rewards. We refer to these game-specific executable models as Game Code World Models (GameCWMs). However, current approaches to generating code world models rely on frontier models and inference-time refinement loops, limiting accessibility and scalability. This work investigates whether GameCWM generation capabilities can be distilled into smaller models through post-training. We introduce: (1) a curated dataset of 30 games spanning perfect and imperfect information games, (2) a verification framework that evaluates generated code against structural and semantic game properties, and (3) a post-training pipeline combining Supervised Fine-Tuning (SFT) with Reinforcement Learning with Verifiable Rewards (RLVR). We experiment with Qwen2.5-3B-Instruct and find that SFT can increase syntactic correctness, while RLVR can improve execution-level adherence to game rules, thereby improving Qwen's ability to generate valid GameCWMs in both perfect and imperfect information games. Overall, our pipeline makes Qwen2.5-3B-Instruct more capable of generating valid GameCWMs, thereby offering a scalable path toward automatic environment generation from natural language.

Related papers

• From Gameplay Traces to Game Mechanics: Causal Induction with Large Language Models [64.43268969806098]
  We investigate Causal Induction: the ability to infer governing laws from observational data.<n>We compare two approaches to VGDL generation: direct code generation from observations, and a two-stage method that first infers a structural causal model (SCM) and then translates it into VGDL.<n>Results show that the SCM-based approach more often produces VGDL descriptions closer to the ground truth than direct generation.
  arXiv  Detail & Related papers  (2026-01-30T08:48:23Z)
• Code World Models for General Game Playing [22.382021070682256]
  We use the Large Language Models to translate natural language rules and game trajectories into a formal, executable world model represented as Python code.<n>This generated model serves as a verifiable simulation engine for high-performance planning algorithms.<n>We find that our method outperforms or matches Gemini 2.5 Pro in 9 out of the 10 considered games.
  arXiv  Detail & Related papers  (2025-10-06T07:16:07Z)
• CWM: An Open-Weights LLM for Research on Code Generation with World Models [78.0342683953353]
  We release Code World Model (CWM) to advance research on code generation with world models.<n>We mid-train CWM on a large amount of observation-action trajectories from Python interpreter and agentic Docker environments.<n>We present first steps of how world models can benefit agentic coding, enable step-by-step simulation of Python code execution, and show early results of how reasoning can benefit the latter.
  arXiv  Detail & Related papers  (2025-09-30T21:47:10Z)
• GVGAI-LLM: Evaluating Large Language Model Agents with Infinite Games [7.594173359523366]
  We introduce GVGAI-LLM, a video game benchmark for evaluating the reasoning and problem-solving capabilities of large language models (LLMs)<n>Built on the General Video Game AI framework, it features a diverse collection of arcade-style games designed to test a model's ability to handle tasks that differ from most existing LLM benchmarks.
  arXiv  Detail & Related papers  (2025-08-11T22:17:07Z)
• IFEvalCode: Controlled Code Generation [69.28317223249358]
  The paper introduces forward and backward constraints generation to improve the instruction-following capabilities of Code LLMs.<n>The authors present IFEvalCode, a multilingual benchmark comprising 1.6K test samples across seven programming languages.
  arXiv  Detail & Related papers  (2025-07-30T08:08:48Z)
• Game-RL: Synthesizing Multimodal Verifiable Game Data to Boost VLMs' General Reasoning [89.93384726755106]
  Vision-language reinforcement learning (RL) has primarily focused on narrow domains.<n>We find video games inherently provide rich visual elements and mechanics that are easy to verify.<n>To fully use the multimodal and verifiable reward in video games, we propose Game-RL.
  arXiv  Detail & Related papers  (2025-05-20T03:47:44Z)
• Measuring General Intelligence with Generated Games [35.118590734217264]
  gg-bench is a collection of game environments designed to evaluate general reasoning capabilities in language models.<n>gg-bench is synthetically generated by (1) using a large language model (LLM) to generate natural language descriptions of novel games, (2) using the LLM to implement each game in code as a Gym environment, and (3) training reinforcement learning (RL) agents via self-play on the generated games.
  arXiv  Detail & Related papers  (2025-05-12T04:01:03Z)
• Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelines [74.42485647685272]
  We focus on Generative Masked Language Models (GMLMs) We train a model to fit conditional probabilities of the data distribution via masking, which are subsequently used as inputs to a Markov Chain to draw samples from the model. We adapt the T5 model for iteratively-refined parallel decoding, achieving 2-3x speedup in machine translation with minimal sacrifice in quality.
  arXiv  Detail & Related papers  (2024-07-22T18:00:00Z)
• Instruction-Driven Game Engines on Large Language Models [59.280666591243154]
  The IDGE project aims to democratize game development by enabling a large language model to follow free-form game rules. We train the IDGE in a curriculum manner that progressively increases the model's exposure to complex scenarios. Our initial progress lies in developing an IDGE for Poker, a universally cherished card game.
  arXiv  Detail & Related papers  (2024-03-30T08:02:16Z)
• Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Models [68.85478477006178]
  We present a Promptable Game Model (PGM) for neural video game simulators. It allows a user to play the game by prompting it with high- and low-level action sequences. Most captivatingly, our PGM unlocks the director's mode, where the game is played by specifying goals for the agents in the form of a prompt. Our method significantly outperforms existing neural video game simulators in terms of rendering quality and unlocks applications beyond the capabilities of the current state of the art.
  arXiv  Detail & Related papers  (2023-03-23T17:43:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.