Related papers: DungeonBench: A Benchmark for Rules-Rich Tactical Reasoning in Dungeons & Dragons Combat

DungeonBench: A Benchmark for Rules-Rich Tactical Reasoning in Dungeons & Dragons Combat

URL: http://arxiv.org/abs/2607.29577v1
Date: Fri, 31 Jul 2026 16:03:38 GMT
Title: DungeonBench: A Benchmark for Rules-Rich Tactical Reasoning in Dungeons & Dragons Combat
Abstract summary: We introduce DungeonBench, a benchmark for tactical reasoning in Dungeons & Dragons combat.<n>At each step, DungeonBench exposes a complete tactical observation, a pending decision, and an indexed list of executable options.<n>The same engine-generated decision stream supports controllers, language-model policies, learned option rankers, and masked-action reinforcement-learning agents.
Score: 0.5352699766206808
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Games and simulators make valuable benchmarks by turning decisions into measurable outcomes, but many current suites under-test rules-rich tactical reasoning: the ability to choose well when geometry, timing, resources, objectives, and rule interactions all matter at once. We introduce DungeonBench, a benchmark for tactical reasoning in Dungeons & Dragons combat, built to cover the vast majority of combat-relevant 2014 System Reference Document content whose effects can be resolved by the simulator while retaining mechanics that simplified combat simulators often abstract away. At each step, DungeonBench exposes a complete tactical observation, a pending decision, and an indexed list of executable options spanning movement, attacks, spells, reactions, objectives, preparation, and scarce resources. The task is to value legal choices whose consequences depend on action economy, creature traits, battlefield geometry, timing windows, and future encounters. DungeonBench has two tracks: Encounter, which evaluates local tactical play in single fights, and Day, which links encounters through persistent hit points, spell slots, consumables, preparation, and short-rest timing, forcing policies to trade off immediate tactical advantage against future survivability. The same engine-generated decision stream supports heuristic controllers, language-model policies, learned option rankers, and masked-action reinforcement-learning agents. We evaluate frontier language-model policies on this shared decision stream. Results show that full tactical observations do not saturate the benchmark: frontier policies often win direct encounters, but linked encounter days expose failures in resource budgeting, rest timing, and rule-aware tactical discipline.

Related papers

Chess on Ice: Curling Tactical Decision-Making via Backward Induction and Deep Reinforcement Learning [10.932188857276605]
We propose a reinforcement learning framework capable of quantitatively evaluating and comparing options in curling.<n>We employ the Deep Deterministic Policy Gradient actor-critic algorithm to exploit the finite-horizon structure of the game.<n>Our experiments show that effective curling strategies can be acquired in a fully self-supervised manner, without any human-annotated data.
arXiv Detail & Related papers (2026-08-03T15:23:42Z)
MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs [54.81359054218573]
We introduce Mindgames, a multi-game arena and evaluation platform for large language models (LLMs)<n>Mindgames provides a unified interaction interface, TrueSkill-based rating, and full trajectory logging across four game environments.<n>We analyze 29,571 multi-agent games with turn-level observations, actions, and rewards, together with MG-Ref, a deterministic offline tournament protocol.
arXiv Detail & Related papers (2026-05-28T07:33:47Z)
GenTac: Generative Modeling and Forecasting of Soccer Tactics [66.35532694378146]
GenTac conceptualizes soccer tactics as a process over continuous multi-player trajectories and discrete semantic events.<n>It supports rich contextual conditioning, including opponent behavior, specific team or league playing styles, and strategic objectives.<n>It can be successfully trained to generalize to other dynamic team sports, including basketball, American football, and ice hockey.
arXiv Detail & Related papers (2026-04-13T17:53:49Z)
GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents [76.60994803070436]
GameWorld is a benchmark for evaluation of Multimodal Large Language Model (MLLM) game agents in browser environments.<n>Two game agent interfaces are studied: (i) computer-use agents that directly emit keyboard and mouse controls, and (ii) generalist multimodal agents that act in a semantic action space.<n>Results across 18 model-interface pairs suggest that even the best performing agent is far from achieving human capabilities on video games.
arXiv Detail & Related papers (2026-04-08T17:49:03Z)
Large Language Models as Pokémon Battle Agents: Strategic Play and Content Generation [4.782714372521615]
Pokémon battles demand reasoning about type matchups, statistical trade-offs, and risk assessment.<n>This work examines whether Large Language Models (LLMs) can serve as competent battle agents.<n>We developed a turn-based Pokémon battle system where LLMs select moves based on battle state rather than pre-programmed logic.
arXiv Detail & Related papers (2025-12-19T07:46:29Z)
Monopoly Deal: A Benchmark Environment for Bounded One-Sided Response Games [0.0]
Card games are widely used to study sequential decision-making under uncertainty.<n>We term games featuring bounded one-sided response games (BORGs)<n>We introduce a modified version of Monopoly Deal as a benchmark environment that isolates this dynamic.
arXiv Detail & Related papers (2025-10-29T01:38:19Z)
Beyond Survival: Evaluating LLMs in Social Deduction Games with Human-Aligned Strategies [54.08697738311866]
Social deduction games like Werewolf combine language, reasoning, and strategy.<n>We curate a high-quality, human-verified multimodal Werewolf dataset containing over 100 hours of video, 32.4M utterance tokens, and 15 rule variants.<n>We propose a novel strategy-alignment evaluation that leverages the winning faction's strategies as ground truth in two stages.
arXiv Detail & Related papers (2025-10-13T13:33:30Z)
LM Fight Arena: Benchmarking Large Multimodal Models via Game Competition [104.81487689011341]
We introduce LM Fight Arena, a novel framework that evaluates large multimodal models in Mortal Kombat II.<n>Unlike static evaluations, LM Fight Arena provides a fully automated, reproducible, and objective assessment of an LMM's strategic reasoning capabilities.
arXiv Detail & Related papers (2025-10-10T02:19:21Z)
NTRL: Encounter Generation via Reinforcement Learning for Dynamic Difficulty Adjustment in Dungeons and Dragons [8.856568375969848]
Encounter Generation via Reinforcement Learning (NTRL) is a novel approach that automates Dynamic Difficulty Adjustment (DDA) in Dungeons & Dragons (D&D)<n>By framing the problem as a contextual bandit, NTRL generates encounters based on real-time party members attributes.<n>In comparison with classic DMs, NTRL iteratively optimize encounters to extend combat longevity (+200%), increases damage dealt to party members, reducing post-combat hit points, and raises the number of player deaths.
arXiv Detail & Related papers (2025-06-24T11:34:01Z)
Efficient exploration of zero-sum stochastic games [83.28949556413717]
We investigate the increasingly important and common game-solving setting where we do not have an explicit description of the game but only oracle access to it through gameplay. During a limited-duration learning phase, the algorithm can control the actions of both players in order to try to learn the game and how to play it well. Our motivation is to quickly learn strategies that have low exploitability in situations where evaluating the payoffs of a queried strategy profile is costly.
arXiv Detail & Related papers (2020-02-24T20:30:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.