Cooperative Heterogeneous Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2011.00791v1
• Date: Mon, 2 Nov 2020 07:39:09 GMT
• Title: Cooperative Heterogeneous Deep Reinforcement Learning
• Authors: Han Zheng, Pengfei Wei, Jing Jiang, Guodong Long, Qinghua Lu, Chengqi Zhang
• Abstract summary: We present a Cooperative Heterogeneous Deep Reinforcement Learning framework that can learn a policy by integrating the advantages of heterogeneous agents. Global agents are off-policy agents that can utilize experiences from the other agents. Local agents are either on-policy agents or population-based evolutionary (EAs) agents that can explore the local area effectively.
• Score: 47.97582814287474
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Numerous deep reinforcement learning agents have been proposed, and each of them has its strengths and flaws. In this work, we present a Cooperative Heterogeneous Deep Reinforcement Learning (CHDRL) framework that can learn a policy by integrating the advantages of heterogeneous agents. Specifically, we propose a cooperative learning framework that classifies heterogeneous agents into two classes: global agents and local agents. Global agents are off-policy agents that can utilize experiences from the other agents. Local agents are either on-policy agents or population-based evolutionary algorithms (EAs) agents that can explore the local area effectively. We employ global agents, which are sample-efficient, to guide the learning of local agents so that local agents can benefit from sample-efficient agents and simultaneously maintain their advantages, e.g., stability. Global agents also benefit from effective local searches. Experimental studies on a range of continuous control tasks from the Mujoco benchmark show that CHDRL achieves better performance compared with state-of-the-art baselines.

Related papers

• Fully Decentralized Cooperative Multi-Agent Reinforcement Learning: A Survey [48.77342627610471]
  Cooperative multi-agent reinforcement learning is a powerful tool to solve many real-world cooperative tasks. It is challenging to derive algorithms that can converge to the optimal joint policy in a fully decentralized setting.
  arXiv  Detail & Related papers  (2024-01-10T05:07:42Z)
• Deep Multi-Agent Reinforcement Learning for Decentralized Active Hypothesis Testing [11.639503711252663]
  We tackle the multi-agent active hypothesis testing (AHT) problem by introducing a novel algorithm rooted in the framework of deep multi-agent reinforcement learning. We present a comprehensive set of experimental results that effectively showcase the agents' ability to learn collaborative strategies and enhance performance.
  arXiv  Detail & Related papers  (2023-09-14T01:18:04Z)
• Group-Agent Reinforcement Learning [12.915860504511523]
  It can largely benefit the reinforcement learning process of each agent if multiple geographically distributed agents perform their separate RL tasks cooperatively. We propose a distributed RL framework called DDAL (Decentralised Distributed Asynchronous Learning) designed for group-agent reinforcement learning (GARL)
  arXiv  Detail & Related papers  (2022-02-10T16:40:59Z)
• A Privacy-preserving Distributed Training Framework for Cooperative Multi-agent Deep Reinforcement Learning [1.14219428942199]
  We propose a new Deep Neural Network (DNN) architecture with both global NN and local NN, and a distributed training framework. We allow the global weights to be updated by all the collaborator agents while the local weights are only updated by the agent they belong to. Experiments show that the framework can efficiently help agents in the same or similar environments to collaborate in their training process and gain a higher convergence rate and better performance.
  arXiv  Detail & Related papers  (2021-09-30T10:53:34Z)
• Locality Matters: A Scalable Value Decomposition Approach for Cooperative Multi-Agent Reinforcement Learning [52.7873574425376]
  Cooperative multi-agent reinforcement learning (MARL) faces significant scalability issues due to state and action spaces that are exponentially large in the number of agents. We propose a novel, value-based multi-agent algorithm called LOMAQ, which incorporates local rewards in the Training Decentralized Execution paradigm.
  arXiv  Detail & Related papers  (2021-09-22T10:08:15Z)
• Non-local Policy Optimization via Diversity-regularized Collaborative Exploration [45.997521480637836]
  We propose a novel non-local policy optimization framework called Diversity-regularized Collaborative Exploration (DiCE) DiCE utilizes a group of heterogeneous agents to explore the environment simultaneously and share the collected experiences. We implement the framework in both on-policy and off-policy settings and the experimental results show that DiCE can achieve substantial improvement over the baselines.
  arXiv  Detail & Related papers  (2020-06-14T03:31:11Z)
• Learning to Incentivize Other Learning Agents [73.03133692589532]
  We show how to equip RL agents with the ability to give rewards directly to other agents, using a learned incentive function. Such agents significantly outperform standard RL and opponent-shaping agents in challenging general-sum Markov games. Our work points toward more opportunities and challenges along the path to ensure the common good in a multi-agent future.
  arXiv  Detail & Related papers  (2020-06-10T20:12:38Z)
• Scalable Multi-Agent Inverse Reinforcement Learning via Actor-Attention-Critic [54.2180984002807]
  Multi-agent adversarial inverse reinforcement learning (MA-AIRL) is a recent approach that applies single-agent AIRL to multi-agent problems. We propose a multi-agent inverse RL algorithm that is more sample-efficient and scalable than previous works.
  arXiv  Detail & Related papers  (2020-02-24T20:30:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.