DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2106.06135v1
• Date: Fri, 11 Jun 2021 02:45:51 GMT
• Title: DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learning
• Authors: Daochen Zha, Jingru Xie, Wenye Ma, Sheng Zhang, Xiangru Lian, Xia Hu, Ji Liu
• Abstract summary: We propose a conceptually simple yet effective DouDizhu AI system, namely DouZero. DouZero enhances traditional Monte-Carlo methods with deep neural networks, action encoding, and parallel actors. It was ranked the first in the Botzone leaderboard among 344 AI agents.
• Score: 65.00325925262948
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Games are abstractions of the real world, where artificial agents learn to compete and cooperate with other agents. While significant achievements have been made in various perfect- and imperfect-information games, DouDizhu (a.k.a. Fighting the Landlord), a three-player card game, is still unsolved. DouDizhu is a very challenging domain with competition, collaboration, imperfect information, large state space, and particularly a massive set of possible actions where the legal actions vary significantly from turn to turn. Unfortunately, modern reinforcement learning algorithms mainly focus on simple and small action spaces, and not surprisingly, are shown not to make satisfactory progress in DouDizhu. In this work, we propose a conceptually simple yet effective DouDizhu AI system, namely DouZero, which enhances traditional Monte-Carlo methods with deep neural networks, action encoding, and parallel actors. Starting from scratch in a single server with four GPUs, DouZero outperformed all the existing DouDizhu AI programs in days of training and was ranked the first in the Botzone leaderboard among 344 AI agents. Through building DouZero, we show that classic Monte-Carlo methods can be made to deliver strong results in a hard domain with a complex action space. The code and an online demo are released at https://github.com/kwai/DouZero with the hope that this insight could motivate future work.

Related papers

• AlphaDou: High-Performance End-to-End Doudizhu AI Integrating Bidding [6.177038245239759]
  This paper modifies the Deep Monte Carlo algorithm framework by using reinforcement learning to obtain a neural network that simultaneously estimates win rates and expectations. The modified algorithm enables the AI to perform the full range of tasks in the Doudizhu game, including bidding and cardplay.
  arXiv  Detail & Related papers  (2024-07-14T17:32:36Z)
• DouRN: Improving DouZero by Residual Neural Networks [1.6013543712340956]
  Doudizhu is a card game that combines elements of cooperation and confrontation, resulting in a large state and action space. In 2021, a Doudizhu program called DouZero surpassed previous models without prior knowledge by utilizing traditional Monte Carlo methods and multilayer perceptrons. Our findings demonstrate that this model significantly improves the winning rate within the same training time.
  arXiv  Detail & Related papers  (2024-03-21T03:25:49Z)
• DanZero+: Dominating the GuanDan Game through Reinforcement Learning [95.90682269990705]
  We develop an AI program for an exceptionally complex and popular card game called GuanDan. We first put forward an AI program named DanZero for this game. In order to further enhance the AI's capabilities, we apply policy-based reinforcement learning algorithm to GuanDan.
  arXiv  Detail & Related papers  (2023-12-05T08:07:32Z)
• DanZero: Mastering GuanDan Game with Reinforcement Learning [121.93690719186412]
  Card game AI has always been a hot topic in the research of artificial intelligence. In this paper, we are devoted to developing an AI program for a more complex card game, GuanDan. We propose the first AI program DanZero for GuanDan using reinforcement learning technique.
  arXiv  Detail & Related papers  (2022-10-31T06:29:08Z)
• DouZero+: Improving DouDizhu AI by Opponent Modeling and Coach-guided Learning [121.93690719186412]
  DouDizhu, a popular card game in China, is very challenging due to the imperfect information, large state space, elements of collaboration and a massive number of possible moves from turn to turn. Recently, a DouDizhu AI system called DouZero has been proposed. Trained using traditional Monte Carlo method with deep neural networks and self-play procedure without the abstraction of human prior knowledge. In this work, we propose to enhance DouZero by introducing opponent modeling into DouZero. Besides, we propose a novel coach network to further boost the performance of DouZero and accelerate its training process.
  arXiv  Detail & Related papers  (2022-04-06T03:18:17Z)
• PerfectDou: Dominating DouDizhu with Perfect Information Distillation [51.069043489706836]
  We propose PerfectDou, a state-of-the-art DouDizhu AI system that dominates the game. In experiments we show how and why PerfectDou beats all existing AI programs, and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-03-30T15:37:57Z)
• Combining Off and On-Policy Training in Model-Based Reinforcement Learning [77.34726150561087]
  We propose a way to obtain off-policy targets using data from simulated games in MuZero. Our results show that these targets speed up the training process and lead to faster convergence and higher rewards.
  arXiv  Detail & Related papers  (2021-02-24T10:47:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.