TStarBot-X: An Open-Sourced and Comprehensive Study for Efficient League Training in StarCraft II Full Game

• URL: http://arxiv.org/abs/2011.13729v2
• Date: Fri, 30 Apr 2021 08:31:32 GMT
• Title: TStarBot-X: An Open-Sourced and Comprehensive Study for Efficient League Training in StarCraft II Full Game
• Authors: Lei Han, Jiechao Xiong, Peng Sun, Xinghai Sun, Meng Fang, Qingwei Guo, Qiaobo Chen, Tengfei Shi, Hongsheng Yu, Xipeng Wu, Zhengyou Zhang
• Abstract summary: Recently, Google's DeepMind announced AlphaStar, a grandmaster level AI in StarCraft II that can play with humans using comparable action space and operations. In this paper, we introduce a new AI agent, named TStarBot-X, that is trained under orders of less computations and can play competitively with expert human players.
• Score: 25.248034258354533
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: StarCraft, one of the most difficult esport games with long-standing history of professional tournaments, has attracted generations of players and fans, and also, intense attentions in artificial intelligence research. Recently, Google's DeepMind announced AlphaStar, a grandmaster level AI in StarCraft II that can play with humans using comparable action space and operations. In this paper, we introduce a new AI agent, named TStarBot-X, that is trained under orders of less computations and can play competitively with expert human players. TStarBot-X takes advantage of important techniques introduced in AlphaStar, and also benefits from substantial innovations including new league training methods, novel multi-agent roles, rule-guided policy search, stabilized policy improvement, lightweight neural network architecture, and importance sampling in imitation learning, etc. We show that with orders of less computation scale, a faithful reimplementation of AlphaStar's methods can not succeed and the proposed techniques are necessary to ensure TStarBot-X's competitive performance. We reveal all technical details that are complementary to those mentioned in AlphaStar, showing the most sensitive parts in league training, reinforcement learning and imitation learning that affect the performance of the agents. Most importantly, this is an open-sourced study that all codes and resources (including the trained model parameters) are publicly accessible via \url{https://github.com/tencent-ailab/tleague_projpage}. We expect this study could be beneficial for both academic and industrial future research in solving complex problems like StarCraft, and also, might provide a sparring partner for all StarCraft II players and other AI agents.

Related papers

• DanZero+: Dominating the GuanDan Game through Reinforcement Learning [95.90682269990705]
  We develop an AI program for an exceptionally complex and popular card game called GuanDan. We first put forward an AI program named DanZero for this game. In order to further enhance the AI's capabilities, we apply policy-based reinforcement learning algorithm to GuanDan.
  arXiv  Detail & Related papers  (2023-12-05T08:07:32Z)
• AlphaStar Unplugged: Large-Scale Offline Reinforcement Learning [38.75717733273262]
  StarCraft II is one of the most challenging simulated reinforcement learning environments. Blizzard has released a massive dataset of millions of StarCraft II games played by human players. We define a dataset (a subset of Blizzard's release), tools standardizing an API for machine learning methods, and an evaluation protocol.
  arXiv  Detail & Related papers  (2023-08-07T12:21:37Z)
• Technical Challenges of Deploying Reinforcement Learning Agents for Game Testing in AAA Games [58.720142291102135]
  We describe an effort to add an experimental reinforcement learning system to an existing automated game testing solution based on scripted bots. We show a use-case of leveraging reinforcement learning in game production and cover some of the largest time sinks anyone who wants to make the same journey for their game may encounter. We propose a few research directions that we believe will be valuable and necessary for making machine learning, and especially reinforcement learning, an effective tool in game production.
  arXiv  Detail & Related papers  (2023-07-19T18:19:23Z)
• DanZero: Mastering GuanDan Game with Reinforcement Learning [121.93690719186412]
  Card game AI has always been a hot topic in the research of artificial intelligence. In this paper, we are devoted to developing an AI program for a more complex card game, GuanDan. We propose the first AI program DanZero for GuanDan using reinforcement learning technique.
  arXiv  Detail & Related papers  (2022-10-31T06:29:08Z)
• Mastering the Game of No-Press Diplomacy via Human-Regularized Reinforcement Learning and Planning [95.78031053296513]
  No-press Diplomacy is a complex strategy game involving both cooperation and competition. We introduce a planning algorithm we call DiL-piKL that regularizes a reward-maximizing policy toward a human imitation-learned policy. We show that DiL-piKL can be extended into a self-play reinforcement learning algorithm we call RL-DiL-piKL.
  arXiv  Detail & Related papers  (2022-10-11T14:47:35Z)
• On Efficient Reinforcement Learning for Full-length Game of StarCraft II [21.768578136029987]
  We investigate a hierarchical RL approach involving extracted macro-actions and a hierarchical architecture of neural networks. On a 64x64 map and using restrictive units, we achieve a win rate of 99% against the level-1 built-in AI. We improve our architecture to train the agent against the cheating level AIs and achieve the win rate against the level-8, level-9, and level-10 AIs as 96%, 97%, and 94%, respectively.
  arXiv  Detail & Related papers  (2022-09-23T12:24:21Z)
• Applying supervised and reinforcement learning methods to create neural-network-based agents for playing StarCraft II [0.0]
  We propose a neural network architecture for playing the full two-player match of StarCraft II trained with general-purpose supervised and reinforcement learning. Our implementation achieves a non-trivial performance when compared to the in-game scripted bots.
  arXiv  Detail & Related papers  (2021-09-26T20:08:10Z)
• An Introduction of mini-AlphaStar [22.820438931820764]
  An SC2 agent called AlphaStar is proposed which shows excellent performance, obtaining a high win-rates of 99.8% against Grandmaster level human players. We implemented a mini-scaled version of it called mini-AlphaStar based on their paper and the pseudocode they provided. The objective of mini-AlphaStar is to provide a reproduction of the original AlphaStar and facilitate the future research of RL on large-scale problems.
  arXiv  Detail & Related papers  (2021-04-14T14:31:51Z)
• SCC: an efficient deep reinforcement learning agent mastering the game of StarCraft II [15.612456049715123]
  AlphaStar, the AI that reaches GrandMaster level in StarCraft II, is a remarkable milestone demonstrating what deep reinforcement learning can achieve. We propose a deep reinforcement learning agent, StarCraft Commander ( SCC) SCC demonstrates top human performance defeating GrandMaster players in test matches and top professional players in a live event.
  arXiv  Detail & Related papers  (2020-12-24T08:43:44Z)
• DeepCrawl: Deep Reinforcement Learning for Turn-based Strategy Games [137.86426963572214]
  We introduce DeepCrawl, a fully-playable Roguelike prototype for iOS and Android in which all agents are controlled by policy networks trained using Deep Reinforcement Learning (DRL) Our aim is to understand whether recent advances in DRL can be used to develop convincing behavioral models for non-player characters in videogames.
  arXiv  Detail & Related papers  (2020-12-03T13:53:29Z)
• Suphx: Mastering Mahjong with Deep Reinforcement Learning [114.68233321904623]
  We design an AI for Mahjong, named Suphx, based on deep reinforcement learning with some newly introduced techniques. Suphx has demonstrated stronger performance than most top human players in terms of stable rank. This is the first time that a computer program outperforms most top human players in Mahjong.
  arXiv  Detail & Related papers  (2020-03-30T16:18:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.