GameCraft-Bench: Can Agents Build Playable Games End-to-End in a Real Game Engine?

• URL: http://arxiv.org/abs/2606.17861v1
• Date: Tue, 16 Jun 2026 12:34:39 GMT
• Title: GameCraft-Bench: Can Agents Build Playable Games End-to-End in a Real Game Engine?
• Abstract summary: Game generation takes place within a game engine, where scripts, scenes, assets, rendering, and runtime interactions must jointly produce coherent gameplay.<n>We formalize end-to-end game generation as the problem of producing a complete game artifact.<n>We instantiate this framework as GameCraft-Bench, a benchmark comprising 140 Godot tasks across 15 game families.
• Score: 65.42976417627254
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Game generation is an emerging application of coding agents, requiring models to transform natural-language specifications into playable interactive systems. Unlike traditional coding tasks, game generation takes place within a game engine, where scripts, scenes, assets, rendering, and runtime interactions must jointly produce coherent gameplay. We formalize end-to-end game generation as the problem of producing a complete game artifact that realizes a specification through observable player-game interaction in a target environment. We argue that evaluating this setting requires three desiderata: Engine Grounding, Artifact Completeness, and Interactive Verification. We propose an interaction-grounded evaluation framework that assesses executable gameplay through replayed demonstrations and rubric-guided multimodal judging. We instantiate this framework as GameCraft-Bench, a benchmark comprising 140 Godot tasks across 15 game families. Evaluations of frontier coding agents show that end-to-end game generation remains highly challenging: the strongest agent achieves only 41.46%, and most agents score below 40%. Further analysis reveals that while agents often implement recognizable mechanics, they struggle to deliver complete games with sufficient content, functional visual feedback, and coherent presentation. See https://tongxuluo.github.io/gamecraft-bench-website for demos, code, and data.

Related papers

• GameXpert-Bench: How Far Are Coding Agents from Expert Game Development? [6.806647989372878]
  Recent large language models (LLMs) can operate as coding agents that build complete games from natural language requests.<n>Game development is especially demanding because program logic, visual and audio content, interfaces, interaction and playability must function together in one executable artifact.<n>Our analysis of complete human-agent development trajectories identifies three stages that together span the lifecycle of game development with a coding agent.
  arXiv  Detail & Related papers  (2026-08-22T08:05:16Z)
• StatePlay: State-Aware Game World Models for Mechanics-Consistent Generation [53.07918503798731]
  StatePlay is a novel state-aware game world model that predicts visual content and game states to promote mechanics-consistent generation.<n>Our work highlights the importance of state-aware game world modeling and advances beyond pixel-level realism toward complete and mechanically faithful game generation.
  arXiv  Detail & Related papers  (2026-07-29T10:49:30Z)
• GUI Agents for Continual Game Generation [22.25472214301186]
  We argue that evaluating and improving game generation requires a player.<n>We introduce PlaytestArena, a new evaluation environment that pairs 200 browser-based game generation tasks with rubrics of expected in-play behaviors.<n>We propose Play2Code, where a game agent and a GUI agent operate in a sustained loop with shared memory, turning game generation into a dialogue between coding and playing.
  arXiv  Detail & Related papers  (2026-05-27T10:08:48Z)
• OpenGame: Open Agentic Coding for Games [18.670337884754577]
  OpenGame is the first open-source agentic framework explicitly designed for end-to-end web game creation.<n>Powering this framework is GameCoder-27B, a code LLM specialized for game engine mastery.<n>OpenGame-Bench is an evaluation pipeline that scores agentic game generation along Build Health, Visual Usability, and Intent Alignment.
  arXiv  Detail & Related papers  (2026-04-20T15:17:03Z)
• GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents [76.60994803070436]
  GameWorld is a benchmark for evaluation of Multimodal Large Language Model (MLLM) game agents in browser environments.<n>Two game agent interfaces are studied: (i) computer-use agents that directly emit keyboard and mouse controls, and (ii) generalist multimodal agents that act in a semantic action space.<n>Results across 18 model-interface pairs suggest that even the best performing agent is far from achieving human capabilities on video games.
  arXiv  Detail & Related papers  (2026-04-08T17:49:03Z)
• GameDevBench: Evaluating Agentic Capabilities Through Game Development [49.19956546746812]
  Game development provides such a testbed as agents must navigate large, denses while manipulating intrinsically multimodal assets.<n>We present GameDevBench, the first benchmark for evaluating agents on game development tasks.<n>Agents still struggle with game development, with the best agent solving only 54.5% of tasks.
  arXiv  Detail & Related papers  (2026-02-11T18:15:11Z)
• NitroGen: An Open Foundation Model for Generalist Gaming Agents [101.41866522979548]
  NitroGen is a vision-action foundation model for generalist gaming agents.<n>It is trained on 40,000 hours of gameplay videos across more than 1,000 games.
  arXiv  Detail & Related papers  (2026-01-04T16:24:50Z)
• 90% Faster, 100% Code-Free: MLLM-Driven Zero-Code 3D Game Development [35.877919141619614]
  This paper introduces UniGen, the first end-to-end coordinated multi-agent framework that automates zero-coding development of runnable 3D games from natural language requirements.<n>Results demonstrate that UniGen not only democratizes game creation by requiring no coding from the user, but also reduces development time by 91.4%.
  arXiv  Detail & Related papers  (2025-09-30T12:14:56Z)
• Playable Game Generation [22.17100581717806]
  We propose emphPlayGen, which encompasses game data generation, an autoregressive DiT-based diffusion model, and a playability-based evaluation framework.<n>PlayGen achieves real-time interaction, ensures sufficient visual quality, and provides accurate interactive mechanics simulation.
  arXiv  Detail & Related papers  (2024-12-01T16:53:02Z)
• Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Models [68.85478477006178]
  We present a Promptable Game Model (PGM) for neural video game simulators. It allows a user to play the game by prompting it with high- and low-level action sequences. Most captivatingly, our PGM unlocks the director's mode, where the game is played by specifying goals for the agents in the form of a prompt. Our method significantly outperforms existing neural video game simulators in terms of rendering quality and unlocks applications beyond the capabilities of the current state of the art.
  arXiv  Detail & Related papers  (2023-03-23T17:43:17Z)
• Learning to Simulate Dynamic Environments with GameGAN [109.25308647431952]
  In this paper, we aim to learn a simulator by simply watching an agent interact with an environment. We introduce GameGAN, a generative model that learns to visually imitate a desired game by ingesting screenplay and keyboard actions during training.
  arXiv  Detail & Related papers  (2020-05-25T14:10:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.