TLeague: A Framework for Competitive Self-Play based Distributed Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2011.12895v2
• Date: Mon, 30 Nov 2020 03:23:36 GMT
• Title: TLeague: A Framework for Competitive Self-Play based Distributed Multi-Agent Reinforcement Learning
• Authors: Peng Sun, Jiechao Xiong, Lei Han, Xinghai Sun, Shuxing Li, Jiawei Xu, Meng Fang, Zhengyou Zhang
• Abstract summary: TLeague aims at large-scale training and implements several main-stream-MARL algorithms. We present experiments over StarCraft II, ViZDoom and Pommerman to show the efficiency and effectiveness of TLeague.
• Score: 28.795986840557475
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Competitive Self-Play (CSP) based Multi-Agent Reinforcement Learning (MARL) has shown phenomenal breakthroughs recently. Strong AIs are achieved for several benchmarks, including Dota 2, Glory of Kings, Quake III, StarCraft II, to name a few. Despite the success, the MARL training is extremely data thirsty, requiring typically billions of (if not trillions of) frames be seen from the environment during training in order for learning a high performance agent. This poses non-trivial difficulties for researchers or engineers and prevents the application of MARL to a broader range of real-world problems. To address this issue, in this manuscript we describe a framework, referred to as TLeague, that aims at large-scale training and implements several main-stream CSP-MARL algorithms. The training can be deployed in either a single machine or a cluster of hybrid machines (CPUs and GPUs), where the standard Kubernetes is supported in a cloud native manner. TLeague achieves a high throughput and a reasonable scale-up when performing distributed training. Thanks to the modular design, it is also easy to extend for solving other multi-agent problems or implementing and verifying MARL algorithms. We present experiments over StarCraft II, ViZDoom and Pommerman to show the efficiency and effectiveness of TLeague. The code is open-sourced and available at https://github.com/tencent-ailab/tleague_projpage

Related papers

• FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning [25.857375787748715]
  We present FightLadder, a real-time fighting game platform, to empower competitive MARL research. We provide implementations of state-of-the-art MARL algorithms for competitive games, as well as a set of evaluation metrics. We demonstrate the feasibility of this platform by training a general agent that consistently defeats 12 built-in characters in single-player mode.
  arXiv  Detail & Related papers  (2024-06-04T08:04:23Z)
• MARL-LNS: Cooperative Multi-agent Reinforcement Learning via Large Neighborhoods Search [27.807695570974644]
  We propose a general training framework, MARL-LNS, to address issues by training on alternating subsets of agents. We show that our algorithms can automatically reduce at least 10% of training time while reaching the same final skill level as the original algorithm.
  arXiv  Detail & Related papers  (2024-04-03T22:51:54Z)
• JaxMARL: Multi-Agent RL Environments and Algorithms in JAX [105.343918678781]
  We present JaxMARL, the first open-source, Python-based library that combines GPU-enabled efficiency with support for a large number of commonly used MARL environments. Our experiments show that, in terms of wall clock time, our JAX-based training pipeline is around 14 times faster than existing approaches. We also introduce and benchmark SMAX, a JAX-based approximate reimplementation of the popular StarCraft Multi-Agent Challenge.
  arXiv  Detail & Related papers  (2023-11-16T18:58:43Z)
• Accelerate Multi-Agent Reinforcement Learning in Zero-Sum Games with Subgame Curriculum Learning [65.36326734799587]
  We present a novel subgame curriculum learning framework for zero-sum games. It adopts an adaptive initial state distribution by resetting agents to some previously visited states. We derive a subgame selection metric that approximates the squared distance to NE values.
  arXiv  Detail & Related papers  (2023-10-07T13:09:37Z)
• Benchmarking Robustness and Generalization in Multi-Agent Systems: A Case Study on Neural MMO [50.58083807719749]
  We present the results of the second Neural MMO challenge, hosted at IJCAI 2022, which received 1600+ submissions. This competition targets robustness and generalization in multi-agent systems. We will open-source our benchmark including the environment wrapper, baselines, a visualization tool, and selected policies for further research.
  arXiv  Detail & Related papers  (2023-08-30T07:16:11Z)
• An Empirical Study on Google Research Football Multi-agent Scenarios [30.926070192524193]
  We open-source our training framework Light-MALib which extends the MALib by distributed and asynchronized implementation with additional analytical tools for football games. We provide guidance for building strong football AI with population-based training and release diverse pretrained policies for benchmarking.
  arXiv  Detail & Related papers  (2023-05-16T14:18:53Z)
• Centralized control for multi-agent RL in a complex Real-Time-Strategy game [0.0]
  Multi-agent Reinforcement learning (MARL) studies the behaviour of multiple learning agents that coexist in a shared environment. MARL is more challenging than single-agent RL because it involves more complex learning dynamics. This project provides the end-to-end experience of applying RL in the Lux AI v2 Kaggle competition.
  arXiv  Detail & Related papers  (2023-04-25T17:19:05Z)
• Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning [73.92475751508452]
  Bimanual Dexterous Hands Benchmark (Bi-DexHands) is a simulator that involves two dexterous hands with tens of bimanual manipulation tasks and thousands of target objects. Tasks in Bi-DexHands are designed to match different levels of human motor skills according to cognitive science literature.
  arXiv  Detail & Related papers  (2022-06-17T11:09:06Z)
• TiKick: Toward Playing Multi-agent Football Full Games from Single-agent Demonstrations [31.596018856092513]
  Tikick is the first learning-based AI system that can take over the multi-agent Google Research Football full game. To the best of our knowledge, Tikick is the first learning-based AI system that can take over the multi-agent Google Research Football full game.
  arXiv  Detail & Related papers  (2021-10-09T08:34:58Z)
• MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learning [61.28547338576706]
  Population-based multi-agent reinforcement learning (PB-MARL) refers to the series of methods nested with reinforcement learning (RL) algorithms. We present MALib, a scalable and efficient computing framework for PB-MARL.
  arXiv  Detail & Related papers  (2021-06-05T03:27:08Z)
• Multi-Agent Collaboration via Reward Attribution Decomposition [75.36911959491228]
  We propose Collaborative Q-learning (CollaQ) that achieves state-of-the-art performance in the StarCraft multi-agent challenge. CollaQ is evaluated on various StarCraft Attribution maps and shows that it outperforms existing state-of-the-art techniques.
  arXiv  Detail & Related papers  (2020-10-16T17:42:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.