Hyperparameter Tricks in Multi-Agent Reinforcement Learning: An Empirical Study

• URL: http://arxiv.org/abs/2102.03479v1
• Date: Sat, 6 Feb 2021 02:28:09 GMT
• Title: Hyperparameter Tricks in Multi-Agent Reinforcement Learning: An Empirical Study
• Authors: Jian Hu, Haibin Wu, Seth Austin Harding, Shih-wei Liao
• Abstract summary: We study and compare the state-of-the-art cooperative multi-agent deep reinforcement learning algorithms. QMIX can attain extraordinarily high win rates in all hard and super hard scenarios of StarCraft Multi-Agent Challenge (SMAC) and achieve state-of-the-art (SOTA)
• Score: 5.811502603310249
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, multi-agent deep reinforcement learning has been successfully applied to various complicated scenarios such as computer games and robot swarms. We thoroughly study and compare the state-of-the-art cooperative multi-agent deep reinforcement learning algorithms. Specifically, we investigate the consequences of the "hyperparameter tricks" of QMIX and its improved variants. Our results show that: (1) The significant performance improvements of these variant algorithms come from hyperparameter-level optimizations in their open-source codes (2) After modest tuning and with no changes to the network architecture, QMIX can attain extraordinarily high win rates in all hard and super hard scenarios of StarCraft Multi-Agent Challenge (SMAC) and achieve state-of-the-art (SOTA). In this work, we proposed a reliable QMIX benchmark, which will be of great benefit to subsequent research. Besides, we proposed a hypothesis to explain the excellent performance of QMIX.

Related papers

• MG-Net: Learn to Customize QAOA with Circuit Depth Awareness [51.78425545377329]
  Quantum Approximate Optimization Algorithm (QAOA) and its variants exhibit immense potential in tackling optimization challenges. The requisite circuit depth for satisfactory performance is problem-specific and often exceeds the maximum capability of current quantum devices. We introduce the Mixer Generator Network (MG-Net), a unified deep learning framework adept at dynamically formulating optimal mixer Hamiltonians.
  arXiv  Detail & Related papers  (2024-09-27T12:28:18Z)
• MLQAOA: Graph Learning Accelerated Hybrid Quantum-Classical Multilevel QAOA [0.7560883489000579]
  We introduce a multilevel algorithm reinforced with the spectral graph representation learning-based accelerator to tackle large-scale graph maximum cut instances. We demonstrate the potential of using multilevel QAOA and representation learning-based approaches on very large graphs by achieving high-quality solutions in a much faster time.
  arXiv  Detail & Related papers  (2024-04-22T17:58:13Z)
• Collaborative Intelligent Reflecting Surface Networks with Multi-Agent Reinforcement Learning [63.83425382922157]
  Intelligent reflecting surface (IRS) is envisioned to be widely applied in future wireless networks. In this paper, we investigate a multi-user communication system assisted by cooperative IRS devices with the capability of energy harvesting.
  arXiv  Detail & Related papers  (2022-03-26T20:37:14Z)
• Quantum circuit architecture search on a superconducting processor [56.04169357427682]
  Variational quantum algorithms (VQAs) have shown strong evidences to gain provable computational advantages for diverse fields such as finance, machine learning, and chemistry. However, the ansatz exploited in modern VQAs is incapable of balancing the tradeoff between expressivity and trainability. We demonstrate the first proof-of-principle experiment of applying an efficient automatic ansatz design technique to enhance VQAs on an 8-qubit superconducting quantum processor.
  arXiv  Detail & Related papers  (2022-01-04T01:53:42Z)
• Value Function Factorisation with Hypergraph Convolution for Cooperative Multi-agent Reinforcement Learning [32.768661516953344]
  We propose a method that combines hypergraph convolution with value decomposition. By treating action values as signals, HGCN-Mix aims to explore the relationship between these signals via a self-learning hypergraph. Experimental results present that HGCN-Mix matches or surpasses state-of-the-art techniques in the StarCraft II multi-agent challenge (SMAC) benchmark.
  arXiv  Detail & Related papers  (2021-12-09T08:40:38Z)
• Towards Multi-Agent Reinforcement Learning using Quantum Boltzmann Machines [2.015864965523243]
  We propose an extension to the original concept in order to solve more challenging problems. We add an experience replay buffer and use different networks for approximating the target and policy values. Quantum sampling proves to be a promising method for reinforcement learning tasks, but is currently limited by the QPU size.
  arXiv  Detail & Related papers  (2021-09-22T17:59:24Z)
• Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learning [89.31889875864599]
  We propose an efficient model-based reinforcement learning algorithm for learning in multi-agent systems. Our main theoretical contributions are the first general regret bounds for model-based reinforcement learning for MFC. We provide a practical parametrization of the core optimization problem.
  arXiv  Detail & Related papers  (2021-07-08T18:01:02Z)
• Softmax with Regularization: Better Value Estimation in Multi-Agent Reinforcement Learning [72.28520951105207]
  Overestimation in $Q$-learning is an important problem that has been extensively studied in single-agent reinforcement learning. We propose a novel regularization-based update scheme that penalizes large joint action-values deviating from a baseline. We show that our method provides a consistent performance improvement on a set of challenging StarCraft II micromanagement tasks.
  arXiv  Detail & Related papers  (2021-03-22T14:18:39Z)
• QTRAN++: Improved Value Transformation for Cooperative Multi-Agent Reinforcement Learning [70.382101956278]
  QTRAN is a reinforcement learning algorithm capable of learning the largest class of joint-action value functions. Despite its strong theoretical guarantee, it has shown poor empirical performance in complex environments. We propose a substantially improved version, coined QTRAN++.
  arXiv  Detail & Related papers  (2020-06-22T05:08:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.