DiG-bench: Discovery in Games

• URL: http://arxiv.org/abs/2608.12593v1
• Date: Wed, 12 Aug 2026 21:06:06 GMT
• Title: DiG-bench: Discovery in Games
• Abstract summary: DiG-bench consists of a set of 70 independent games.<n>The levels of the game present a series of challenges to test whether the rules have been discovered.<n>All 70 games were solved by at least one human on first attempt.
• Score: 46.68149875140523
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Discovery---formulating novel generalizations---is a central part of the scientific process. Despite its importance, there is a gap in the current AI benchmark landscape, with few benchmarks directly probing the capacity for discovering new knowledge with experimentation in controlled environments where the objective is unknown. To address this gap, we release a new benchmark: DiG-bench (Discovery in Games). DiG-bench consists of a set of 70 independent games. Each game is encoded as a short string and has unique transformation rules that must be discovered through interaction and experimentation. The levels of the game present a series of challenges to test whether the rules have been discovered, where the win conditions for each level are also unknown. We provide games at seven tiers of difficulty for AI agents. The lowest tier is routinely solvable by multiple models, while the highest tier challenges the best models in agentic harnesses. All 70 games were solved by at least one human on first attempt. A subset of 21 games is released publicly, and the remainder is held private for secure evaluation.

Related papers

• Twin: Playing an Unknown Game with a Test-Time Digital Twin [22.644194631480772]
  We present a Test-time World-model Inference (Twin) system, in which a frontier coding agent writes an executable world model for completing continual learning tasks.<n>Our system constructs them from simulation and interaction alone.<n>Twin clears 179 out of 183 levels (97.8%), and does so more efficiently than humans in 158 out of 179 levels (88.3%)
  arXiv  Detail & Related papers  (2026-08-14T17:06:00Z)
• GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents [76.60994803070436]
  GameWorld is a benchmark for evaluation of Multimodal Large Language Model (MLLM) game agents in browser environments.<n>Two game agent interfaces are studied: (i) computer-use agents that directly emit keyboard and mouse controls, and (ii) generalist multimodal agents that act in a semantic action space.<n>Results across 18 model-interface pairs suggest that even the best performing agent is far from achieving human capabilities on video games.
  arXiv  Detail & Related papers  (2026-04-08T17:49:03Z)
• AI Gamestore: Scalable, Open-Ended Evaluation of Machine General Intelligence with Human Games [63.29377274531968]
  We introduce the AI GameStore, a scalable and open-ended platform to synthesize new representative human games.<n>We generate 100 such games based on the top charts of Apple App Store and Steam, and evaluate seven frontier vision-language models (VLMs) on short episodes of play.<n>The best models achieved less than 10% of the human average score on the majority of the games, and especially struggled with games that challenge world-model learning, memory and planning.
  arXiv  Detail & Related papers  (2026-02-19T18:17:25Z)
• FlashAdventure: A Benchmark for GUI Agents Solving Full Story Arcs in Diverse Adventure Games [56.81554611870848]
  We introduce FlashAdventure, a benchmark of 34 Flash-based adventure games designed to test full story arc completion.<n>We also propose CUA-as-a-Judge, an automated gameplay evaluator, and COAST, an agentic framework leveraging long-term clue memory.<n> Experiments show current GUI agents struggle with full story arcs, while COAST improves milestone completion by bridging the observation-behavior gap.
  arXiv  Detail & Related papers  (2025-09-01T01:33:16Z)
• Securing Equal Share: A Principled Approach for Learning Multiplayer Symmetric Games [21.168085154982712]
  equilibria in multiplayer games are neither unique nor non-exploitable. This paper takes an initial step towards addressing these challenges by focusing on the natural objective of equal share. We design a series of efficient algorithms, inspired by no-regret learning, that provably attain approximate equal share across various settings.
  arXiv  Detail & Related papers  (2024-06-06T15:59:17Z)
• Teamwork under extreme uncertainty: AI for Pokemon ranks 33rd in the world [0.0]
  This paper describes the mechanics of the game and we perform a game analysis. We propose unique AI algorithms based on our understanding that the two biggest challenges in the game are keeping a balanced team and dealing with three sources of uncertainty. Our AI agent performed significantly better than all previous attempts and peaked at the 33rd place in the world, in one of the most popular battle formats, while running on only 4 single socket servers.
  arXiv  Detail & Related papers  (2022-12-27T01:52:52Z)
• CommonsenseQA 2.0: Exposing the Limits of AI through Gamification [126.85096257968414]
  We construct benchmarks that test the abilities of modern natural language understanding models. In this work, we propose gamification as a framework for data construction.
  arXiv  Detail & Related papers  (2022-01-14T06:49:15Z)
• TotalBotWar: A New Pseudo Real-time Multi-action Game Challenge and Competition for AI [62.997667081978825]
  TotalBotWar is a new pseudo real-time multi-action challenge for game AI. The game is based on the popular TotalWar games series where players manage an army to defeat the opponent's one.
  arXiv  Detail & Related papers  (2020-09-18T09:13:56Z)
• Inducing game rules from varying quality game play [0.0]
  General Game Playing (GGP) is a framework in which an artificial intelligence program is required to play a variety of games successfully. IGGP is the problem of inducing general game rules from specific game observations. We use Sancho, the 2014 GGP competition winner, to generate intelligent game traces for a large number of games. We then use the ILP systems, Metagol, Aleph and ILASP to induce game rules from the traces.
  arXiv  Detail & Related papers  (2020-08-04T15:46:57Z)
• Finding Game Levels with the Right Difficulty in a Few Trials through Intelligent Trial-and-Error [16.297059109611798]
  Methods for dynamic difficulty adjustment allow games to be tailored to particular players to maximize their engagement. Current methods often only modify a limited set of game features such as the difficulty of the opponents, or the availability of resources. This paper presents a method that can generate and search for complete levels with a specific target difficulty in only a few trials.
  arXiv  Detail & Related papers  (2020-05-15T17:48:18Z)
• Nmbr9 as a Constraint Programming Challenge [0.0]
  The game Nmbr9 is a solitaire style puzzle game using polyominoes. The rules of the game are simple to explain, but modelling the game effectively using constraint programming is hard. This abstract presents the game, contributes new generalized variants of the game suitable for benchmarking and testing, and describes a model for the presented variants.
  arXiv  Detail & Related papers  (2020-01-13T13:40:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.