GameEngineBench: Evaluating Coding Agents on Real C++ Runtime Environments

• URL: http://arxiv.org/abs/2607.03525v1
• Date: Fri, 03 Jul 2026 17:56:26 GMT
• Title: GameEngineBench: Evaluating Coding Agents on Real C++ Runtime Environments
• Abstract summary: We present GameEngineBench, a benchmark for evaluating coding agents on scoped C++ implementation tasks inside Unreal Engine 5 projects.<n>The evaluation set consists of 110 tasks spanning gameplay mechanics, multiplayer behavior, AI and world orchestration, animation and movement, UI and session code, loading behavior, online-service integration, persistence, data serialization, XR behavior, and rendering-oriented plugins.
• Score: 1.392548092257887
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Game engines provide real-time simulation, rendering, physics, interaction, networking, and asset pipelines, making them valuable not only for games but also for 3D applications in healthcare, robotics, architecture, manufacturing, and related domains. Because game development is where these systems are most mature and publicly available, it offers a practical testbed for evaluating coding agents that must modify C++ code within stateful, interactive, real-time systems. We present GameEngineBench, a benchmark for evaluating coding agents on scoped C++ implementation tasks inside Unreal Engine 5 projects, built from nine real-world game repositories. The evaluation set consists of 110 tasks spanning gameplay mechanics, multiplayer behavior, AI and world orchestration, animation and movement, UI and session code, loading behavior, online-service integration, persistence, data serialization, XR behavior, and rendering-oriented plugins. These tasks require models to make native C++ changes that compile and satisfy behavioral tests within executable Unreal Engine projects. Across twelve evaluated configurations, the strongest model reaches 55.5\% pass@1, while 31 tasks remain unsolved by every configuration. Our results demonstrate that frontier coding agents continue to struggle with deeply integrated C++ development for real-time interactive software, highlighting game-engine benchmarks as a valuable complement to existing software engineering evaluations.

Related papers

• AI for Games in the Foundation Model Era [36.41163570360799]
  Foundation models, alongside advances in learned game-world models, are reshaping AI across the game lifecycle.<n>We organize the literature into six roles according to the immediate use of AI output.<n>For each role, we examine what structure is supplied by the game or workflow, what AI learns or produces, which capabilities and artifacts transfer across settings and roles, and what evidence supports the claims.
  arXiv  Detail & Related papers  (2026-09-15T06:00:47Z)
• GameXpert-Bench: How Far Are Coding Agents from Expert Game Development? [6.806647989372878]
  Recent large language models (LLMs) can operate as coding agents that build complete games from natural language requests.<n>Game development is especially demanding because program logic, visual and audio content, interfaces, interaction and playability must function together in one executable artifact.<n>Our analysis of complete human-agent development trajectories identifies three stages that together span the lifecycle of game development with a coding agent.
  arXiv  Detail & Related papers  (2026-08-22T08:05:16Z)
• GameCraft-Bench: Can Agents Build Playable Games End-to-End in a Real Game Engine? [65.42976417627254]
  Game generation takes place within a game engine, where scripts, scenes, assets, rendering, and runtime interactions must jointly produce coherent gameplay.<n>We formalize end-to-end game generation as the problem of producing a complete game artifact.<n>We instantiate this framework as GameCraft-Bench, a benchmark comprising 140 Godot tasks across 15 game families.
  arXiv  Detail & Related papers  (2026-06-16T12:34:39Z)
• SceneCode: Executable World Programs for Editable Indoor Scenes with Articulated Objects [69.20984454755512]
  Indoor scene synthesis underpins embodied AI, robotic manipulation, and simulation-based policy evaluation.<n>Existing pipelines represent generated content as static meshes and inherit articulation only from curated asset libraries.<n>We present SceneCode, a framework that compiles a natural language prompt into an executable, code-driven indoor world.
  arXiv  Detail & Related papers  (2026-05-19T09:31:04Z)
• GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents [76.60994803070436]
  GameWorld is a benchmark for evaluation of Multimodal Large Language Model (MLLM) game agents in browser environments.<n>Two game agent interfaces are studied: (i) computer-use agents that directly emit keyboard and mouse controls, and (ii) generalist multimodal agents that act in a semantic action space.<n>Results across 18 model-interface pairs suggest that even the best performing agent is far from achieving human capabilities on video games.
  arXiv  Detail & Related papers  (2026-04-08T17:49:03Z)
• GameDevBench: Evaluating Agentic Capabilities Through Game Development [49.19956546746812]
  Game development provides such a testbed as agents must navigate large, denses while manipulating intrinsically multimodal assets.<n>We present GameDevBench, the first benchmark for evaluating agents on game development tasks.<n>Agents still struggle with game development, with the best agent solving only 54.5% of tasks.
  arXiv  Detail & Related papers  (2026-02-11T18:15:11Z)
• Seed3D 1.0: From Images to High-Fidelity Simulation-Ready 3D Assets [63.67760219308476]
  We present Seed3D 1.0, a foundation model that generates simulation-ready 3D assets from single images.<n>Unlike existing 3D generation models, our system produces assets with accurate geometry, well-aligned textures, and realistic physically-based materials.
  arXiv  Detail & Related papers  (2025-10-22T18:16:32Z)
• 90% Faster, 100% Code-Free: MLLM-Driven Zero-Code 3D Game Development [35.877919141619614]
  This paper introduces UniGen, the first end-to-end coordinated multi-agent framework that automates zero-coding development of runnable 3D games from natural language requirements.<n>Results demonstrate that UniGen not only democratizes game creation by requiring no coding from the user, but also reduces development time by 91.4%.
  arXiv  Detail & Related papers  (2025-09-30T12:14:56Z)
• SyDRA: An Approach to Understand Game Engine Architecture [2.5491998280343555]
  We present the Subsystem-Dependency Recovery Approach (SyDRA) to help developers understand game engine architecture. SyDRA helps game engine developers understand game engine architecture and make informed game engine development choices. We show that SyDRA enables developers to complete tasks related to architectural understanding and impact analysis in less time and with higher correctness than without these models.
  arXiv  Detail & Related papers  (2024-06-08T14:40:29Z)
• Which architecture should be implemented to manage data from the real world, in an Unreal Engine 5 simulator and in the context of mixed reality? [0.0]
  This paper gives a detailed analysis of the issue, at both theoretical and operational level. The C++ system is reviewed in details as well as the third-party library: pitfalls to be avoided are shown. The last chapter proposes a generic architecture, useful in large-scale industrial 3D applications.
  arXiv  Detail & Related papers  (2023-05-16T07:51:54Z)
• Learning to Simulate Dynamic Environments with GameGAN [109.25308647431952]
  In this paper, we aim to learn a simulator by simply watching an agent interact with an environment. We introduce GameGAN, a generative model that learns to visually imitate a desired game by ingesting screenplay and keyboard actions during training.
  arXiv  Detail & Related papers  (2020-05-25T14:10:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.