Gym-$\mu$RTS: Toward Affordable Full Game Real-time Strategy Games Research with Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2105.13807v1
• Date: Fri, 21 May 2021 20:13:35 GMT
• Title: Gym-$\mu$RTS: Toward Affordable Full Game Real-time Strategy Games Research with Deep Reinforcement Learning
• Authors: Shengyi Huang, Santiago Onta\~n\'on, Chris Bamford, Lukasz Grela
• Abstract summary: We introduce Gym-$mu$RTS as a fast-to-run RL environment for full-game RTS research. We present a collection of techniques to scale DRL to play full-game $mu$RTS.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, researchers have achieved great success in applying Deep Reinforcement Learning (DRL) algorithms to Real-time Strategy (RTS) games, creating strong autonomous agents that could defeat professional players in StarCraft~II. However, existing approaches to tackle full games have high computational costs, usually requiring the use of thousands of GPUs and CPUs for weeks. This paper has two main contributions to address this issue: 1) We introduce Gym-$\mu$RTS (pronounced "gym-micro-RTS") as a fast-to-run RL environment for full-game RTS research and 2) we present a collection of techniques to scale DRL to play full-game $\mu$RTS as well as ablation studies to demonstrate their empirical importance. Our best-trained bot can defeat every $\mu$RTS bot we tested from the past $\mu$RTS competitions when working in a single-map setting, resulting in a state-of-the-art DRL agent while only taking about 60 hours of training using a single machine (one GPU, three vCPU, 16GB RAM).

Related papers

• A Competition Winning Deep Reinforcement Learning Agent in microRTS [0.0]
  RAISocketAI is the first Deep Reinforcement Learning (DRL) agent to win the IEEE microRTS competition. In a benchmark without performance constraints, RAISocketAI regularly defeated the two prior competition winners. Iteratively fine-tuning the base policy and transfer learning to specific maps were critical to RAISocketAI's winning performance.
  arXiv  Detail & Related papers  (2024-02-12T23:08:17Z)
• DanZero+: Dominating the GuanDan Game through Reinforcement Learning [95.90682269990705]
  We develop an AI program for an exceptionally complex and popular card game called GuanDan. We first put forward an AI program named DanZero for this game. In order to further enhance the AI's capabilities, we apply policy-based reinforcement learning algorithm to GuanDan.
  arXiv  Detail & Related papers  (2023-12-05T08:07:32Z)
• Technical Challenges of Deploying Reinforcement Learning Agents for Game Testing in AAA Games [58.720142291102135]
  We describe an effort to add an experimental reinforcement learning system to an existing automated game testing solution based on scripted bots. We show a use-case of leveraging reinforcement learning in game production and cover some of the largest time sinks anyone who wants to make the same journey for their game may encounter. We propose a few research directions that we believe will be valuable and necessary for making machine learning, and especially reinforcement learning, an effective tool in game production.
  arXiv  Detail & Related papers  (2023-07-19T18:19:23Z)
• Mastering the Game of No-Press Diplomacy via Human-Regularized Reinforcement Learning and Planning [95.78031053296513]
  No-press Diplomacy is a complex strategy game involving both cooperation and competition. We introduce a planning algorithm we call DiL-piKL that regularizes a reward-maximizing policy toward a human imitation-learned policy. We show that DiL-piKL can be extended into a self-play reinforcement learning algorithm we call RL-DiL-piKL.
  arXiv  Detail & Related papers  (2022-10-11T14:47:35Z)
• On Efficient Reinforcement Learning for Full-length Game of StarCraft II [21.768578136029987]
  We investigate a hierarchical RL approach involving extracted macro-actions and a hierarchical architecture of neural networks. On a 64x64 map and using restrictive units, we achieve a win rate of 99% against the level-1 built-in AI. We improve our architecture to train the agent against the cheating level AIs and achieve the win rate against the level-8, level-9, and level-10 AIs as 96%, 97%, and 94%, respectively.
  arXiv  Detail & Related papers  (2022-09-23T12:24:21Z)
• Applying supervised and reinforcement learning methods to create neural-network-based agents for playing StarCraft II [0.0]
  We propose a neural network architecture for playing the full two-player match of StarCraft II trained with general-purpose supervised and reinforcement learning. Our implementation achieves a non-trivial performance when compared to the in-game scripted bots.
  arXiv  Detail & Related papers  (2021-09-26T20:08:10Z)
• SCC: an efficient deep reinforcement learning agent mastering the game of StarCraft II [15.612456049715123]
  AlphaStar, the AI that reaches GrandMaster level in StarCraft II, is a remarkable milestone demonstrating what deep reinforcement learning can achieve. We propose a deep reinforcement learning agent, StarCraft Commander ( SCC) SCC demonstrates top human performance defeating GrandMaster players in test matches and top professional players in a live event.
  arXiv  Detail & Related papers  (2020-12-24T08:43:44Z)
• DeepCrawl: Deep Reinforcement Learning for Turn-based Strategy Games [137.86426963572214]
  We introduce DeepCrawl, a fully-playable Roguelike prototype for iOS and Android in which all agents are controlled by policy networks trained using Deep Reinforcement Learning (DRL) Our aim is to understand whether recent advances in DRL can be used to develop convincing behavioral models for non-player characters in videogames.
  arXiv  Detail & Related papers  (2020-12-03T13:53:29Z)
• The NetHack Learning Environment [79.06395964379107]
  We present the NetHack Learning Environment (NLE), a procedurally generated rogue-like environment for Reinforcement Learning research. We argue that NetHack is sufficiently complex to drive long-term research on problems such as exploration, planning, skill acquisition, and language-conditioned RL. We demonstrate empirical success for early stages of the game using a distributed Deep RL baseline and Random Network Distillation exploration.
  arXiv  Detail & Related papers  (2020-06-24T14:12:56Z)
• Deep RL Agent for a Real-Time Action Strategy Game [0.3867363075280543]
  We introduce a reinforcement learning environment based on Heroic - Magic Duel, a 1 v 1 action strategy game. Our main contribution is a deep reinforcement learning agent playing the game at a competitive level. Our best self-play agent, obtains around $65%$ win rate against the existing AI and over $50%$ win rate against a top human player.
  arXiv  Detail & Related papers  (2020-02-15T01:09:56Z)
• Provable Self-Play Algorithms for Competitive Reinforcement Learning [48.12602400021397]
  We study self-play in competitive reinforcement learning under the setting of Markov games. We show that a self-play algorithm achieves regret $tildemathcalO(sqrtT)$ after playing $T$ steps of the game. We also introduce an explore-then-exploit style algorithm, which achieves a slightly worse regret $tildemathcalO(T2/3)$, but is guaranteed to run in time even in the worst case.
  arXiv  Detail & Related papers  (2020-02-10T18:44:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.