A Local Information Aggregation based Multi-Agent Reinforcement Learning for Robot Swarm Dynamic Task Allocation

• URL: http://arxiv.org/abs/2411.19526v1
• Date: Fri, 29 Nov 2024 07:53:05 GMT
• Title: A Local Information Aggregation based Multi-Agent Reinforcement Learning for Robot Swarm Dynamic Task Allocation
• Authors: Yang Lv, Jinlong Lei, Peng Yi,
• Abstract summary: We introduce a novel framework using a decentralized partially observable Markov decision process (Dec_POMDP) At the core of our methodology is the Local Information Aggregation Multi-Agent Deep Deterministic Policy Gradient (LIA_MADDPG) algorithm. Our empirical evaluations show that the LIA module can be seamlessly integrated into various CTDE-based MARL methods.
• Score: 4.144893164317513
• License:
• Abstract: In this paper, we explore how to optimize task allocation for robot swarms in dynamic environments, emphasizing the necessity of formulating robust, flexible, and scalable strategies for robot cooperation. We introduce a novel framework using a decentralized partially observable Markov decision process (Dec_POMDP), specifically designed for distributed robot swarm networks. At the core of our methodology is the Local Information Aggregation Multi-Agent Deep Deterministic Policy Gradient (LIA_MADDPG) algorithm, which merges centralized training with distributed execution (CTDE). During the centralized training phase, a local information aggregation (LIA) module is meticulously designed to gather critical data from neighboring robots, enhancing decision-making efficiency. In the distributed execution phase, a strategy improvement method is proposed to dynamically adjust task allocation based on changing and partially observable environmental conditions. Our empirical evaluations show that the LIA module can be seamlessly integrated into various CTDE-based MARL methods, significantly enhancing their performance. Additionally, by comparing LIA_MADDPG with six conventional reinforcement learning algorithms and a heuristic algorithm, we demonstrate its superior scalability, rapid adaptation to environmental changes, and ability to maintain both stability and convergence speed. These results underscore LIA_MADDPG's outstanding performance and its potential to significantly improve dynamic task allocation in robot swarms through enhanced local collaboration and adaptive strategy execution.

Related papers

• Cluster-Based Multi-Agent Task Scheduling for Space-Air-Ground Integrated Networks [60.085771314013044]
  Low-altitude economy holds significant potential for development in areas such as communication and sensing. We propose a Clustering-based Multi-agent Deep Deterministic Policy Gradient (CMADDPG) algorithm to address the multi-UAV cooperative task scheduling challenges in SAGIN.
  arXiv  Detail & Related papers  (2024-12-14T06:17:33Z)
• Online Parallel Multi-Task Relationship Learning via Alternating Direction Method of Multipliers [37.859185005986056]
  Online multi-task learning (OMTL) enhances streaming data processing by leveraging the inherent relations among multiple tasks. This study proposes a novel OMTL framework based on the alternating direction multiplier method (ADMM), a recent breakthrough in optimization suitable for the distributed computing environment.
  arXiv  Detail & Related papers  (2024-11-09T10:20:13Z)
• Cooperative and Asynchronous Transformer-based Mission Planning for Heterogeneous Teams of Mobile Robots [1.1049608786515839]
  We propose a Cooperative and Asynchronous Transformer-based Mission Planning (CATMiP) framework to coordinate distributed decision making among agents. We evaluate CATMiP in a 2D grid-world simulation environment and compare its performance against planning-based exploration methods.
  arXiv  Detail & Related papers  (2024-10-08T21:14:09Z)
• Distributed Autonomous Swarm Formation for Dynamic Network Bridging [40.27919181139919]
  We formulate the problem of dynamic network bridging in a novel Decentralized Partially Observable Markov Decision Process (Dec-POMDP) We propose a Multi-Agent Reinforcement Learning (MARL) approach for the problem based on Graph Convolutional Reinforcement Learning (DGN) The proposed method is evaluated in a simulated environment and compared to a centralized baseline showing promising results.
  arXiv  Detail & Related papers  (2024-04-02T01:45:03Z)
• Machine Learning Insides OptVerse AI Solver: Design Principles and Applications [74.67495900436728]
  We present a comprehensive study on the integration of machine learning (ML) techniques into Huawei Cloud's OptVerse AI solver. We showcase our methods for generating complex SAT and MILP instances utilizing generative models that mirror multifaceted structures of real-world problem. We detail the incorporation of state-of-the-art parameter tuning algorithms which markedly elevate solver performance.
  arXiv  Detail & Related papers  (2024-01-11T15:02:15Z)
• Macro-Action-Based Multi-Agent/Robot Deep Reinforcement Learning under Partial Observability [4.111899441919164]
  State-of-the-art multi-agent reinforcement learning (MARL) methods have provided promising solutions to a variety of complex problems. We first propose a group of value-based RL approaches for MacDec-POMDPs. We formulate a set of macro-action-based policy gradient algorithms under the three training paradigms.
  arXiv  Detail & Related papers  (2022-09-20T21:13:51Z)
• Learning Cooperation and Online Planning Through Simulation and Graph Convolutional Network [5.505634045241288]
  We introduce a simulation based online planning algorithm, that we call SiCLOP, for multi-agent cooperative environments. Specifically, SiCLOP tailors Monte Carlo Tree Search (MCTS) and uses Coordination Graph (CG) and Graph Neural Network (GCN) to learn cooperation. It also improves scalability through an effective pruning of action space.
  arXiv  Detail & Related papers  (2021-10-16T05:54:32Z)
• Graph Neural Networks for Decentralized Multi-Robot Submodular Action Selection [101.38634057635373]
  We focus on applications where robots are required to jointly select actions to maximize team submodular objectives. We propose a general-purpose learning architecture towards submodular at scale, with decentralized communications. We demonstrate the performance of our GNN-based learning approach in a scenario of active target coverage with large networks of robots.
  arXiv  Detail & Related papers  (2021-05-18T15:32:07Z)
• Learning to Continuously Optimize Wireless Resource in a Dynamic Environment: A Bilevel Optimization Perspective [52.497514255040514]
  This work develops a new approach that enables data-driven methods to continuously learn and optimize resource allocation strategies in a dynamic environment. We propose to build the notion of continual learning into wireless system design, so that the learning model can incrementally adapt to the new episodes. Our design is based on a novel bilevel optimization formulation which ensures certain fairness" across different data samples.
  arXiv  Detail & Related papers  (2021-05-03T07:23:39Z)
• F2A2: Flexible Fully-decentralized Approximate Actor-critic for Cooperative Multi-agent Reinforcement Learning [110.35516334788687]
  Decentralized multi-agent reinforcement learning algorithms are sometimes unpractical in complicated applications. We propose a flexible fully decentralized actor-critic MARL framework, which can handle large-scale general cooperative multi-agent setting. Our framework can achieve scalability and stability for large-scale environment and reduce information transmission.
  arXiv  Detail & Related papers  (2020-04-17T14:56:29Z)
• Decentralized MCTS via Learned Teammate Models [89.24858306636816]
  We present a trainable online decentralized planning algorithm based on decentralized Monte Carlo Tree Search. We show that deep learning and convolutional neural networks can be employed to produce accurate policy approximators.
  arXiv  Detail & Related papers  (2020-03-19T13:10:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.