Multi-Agent Deep Reinforcement Learning for Distributed Satellite Routing

• URL: http://arxiv.org/abs/2402.17666v1
• Date: Tue, 27 Feb 2024 16:36:53 GMT
• Title: Multi-Agent Deep Reinforcement Learning for Distributed Satellite Routing
• Authors: Federico Lozano-Cuadra, Beatriz Soret
• Abstract summary: This paper introduces a Multi-Agent Deep Reinforcement Learning (MA-DRL) approach for routing in Low Earth Orbit Satellite Constellations (LSatCs) Results show that MA-DRL efficiently learns optimal routes offline that are then loaded for an efficient distributed routing online.
• Score: 7.793857269225969
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper introduces a Multi-Agent Deep Reinforcement Learning (MA-DRL) approach for routing in Low Earth Orbit Satellite Constellations (LSatCs). Each satellite is an independent decision-making agent with a partial knowledge of the environment, and supported by feedback received from the nearby agents. Building on our previous work that introduced a Q-routing solution, the contribution of this paper is to extend it to a deep learning framework able to quickly adapt to the network and traffic changes, and based on two phases: (1) An offline exploration learning phase that relies on a global Deep Neural Network (DNN) to learn the optimal paths at each possible position and congestion level; (2) An online exploitation phase with local, on-board, pre-trained DNNs. Results show that MA-DRL efficiently learns optimal routes offline that are then loaded for an efficient distributed routing online.

Related papers

• Intelligent Routing Algorithm over SDN: Reusable Reinforcement Learning Approach [1.799933345199395]
  We develop a reusable RL-aware, reusable routing algorithm, RLSR-Routing over SDN. Our algorithm shows better performance in terms of load balancing than the traditional approaches. It also has faster convergence than the non-reusable RL approach when finding paths for multiple traffic demands.
  arXiv  Detail & Related papers  (2024-09-23T17:15:24Z)
• Cooperative Path Planning with Asynchronous Multiagent Reinforcement Learning [4.640948267127441]
  shortest path problem (SPP) with multiple source-destination pairs (MSD) In this paper, we study the shortest path problem (SPP) with multiple source-destination pairs (MSD), namely MSD-SPP, to minimize average travel time of all shortest paths.
  arXiv  Detail & Related papers  (2024-09-01T15:48:14Z)
• Continual Deep Reinforcement Learning for Decentralized Satellite Routing [34.67962234401005]
  This paper introduces a full solution for decentralized routing in Low Earth Orbit satellite constellations based on continual Deep Reinforcement Learning (DRL) We follow a multi-agent approach, where each satellite acts as an independent decision-making agent. Our solution adapts well to congestion conditions and exploits less loaded paths.
  arXiv  Detail & Related papers  (2024-05-20T18:12:36Z)
• An Efficient Learning-based Solver Comparable to Metaheuristics for the Capacitated Arc Routing Problem [67.92544792239086]
  We introduce an NN-based solver to significantly narrow the gap with advanced metaheuristics. First, we propose direction-aware facilitating attention model (DaAM) to incorporate directionality into the embedding process. Second, we design a supervised reinforcement learning scheme that involves supervised pre-training to establish a robust initial policy.
  arXiv  Detail & Related papers  (2024-03-11T02:17:42Z)
• MARLIN: Soft Actor-Critic based Reinforcement Learning for Congestion Control in Real Networks [63.24965775030673]
  We propose a novel Reinforcement Learning (RL) approach to design generic Congestion Control (CC) algorithms. Our solution, MARLIN, uses the Soft Actor-Critic algorithm to maximize both entropy and return. We trained MARLIN on a real network with varying background traffic patterns to overcome the sim-to-real mismatch.
  arXiv  Detail & Related papers  (2023-02-02T18:27:20Z)
• Network Slicing via Transfer Learning aided Distributed Deep Reinforcement Learning [7.126310378721161]
  We propose a novel transfer learning (TL) aided multi-agent deep reinforcement learning (MADRL) approach with inter-agent similarity analysis for inter-cell inter-slice resource partitioning. We show that our approach outperforms the state-of-the-art solutions in terms of performance, convergence speed and sample efficiency.
  arXiv  Detail & Related papers  (2023-01-09T10:55:13Z)
• Optimization for Master-UAV-powered Auxiliary-Aerial-IRS-assisted IoT Networks: An Option-based Multi-agent Hierarchical Deep Reinforcement Learning Approach [56.84948632954274]
  This paper investigates a master unmanned aerial vehicle (MUAV)-powered Internet of Things (IoT) network. We propose using a rechargeable auxiliary UAV (AUAV) equipped with an intelligent reflecting surface (IRS) to enhance the communication signals from the MUAV. Under the proposed model, we investigate the optimal collaboration strategy of these energy-limited UAVs to maximize the accumulated throughput of the IoT network.
  arXiv  Detail & Related papers  (2021-12-20T15:45:28Z)
• Deep Reinforcement Learning Aided Packet-Routing For Aeronautical Ad-Hoc Networks Formed by Passenger Planes [99.54065757867554]
  We invoke deep reinforcement learning for routing in AANETs aiming at minimizing the end-to-end (E2E) delay. A deep Q-network (DQN) is conceived for capturing the relationship between the optimal routing decision and the local geographic information observed by the forwarding node. We further exploit the knowledge concerning the system's dynamics by using a deep value network (DVN) conceived with a feedback mechanism.
  arXiv  Detail & Related papers  (2021-10-28T14:18:56Z)
• Deep Learning Aided Packet Routing in Aeronautical Ad-Hoc Networks Relying on Real Flight Data: From Single-Objective to Near-Pareto Multi-Objective Optimization [79.96177511319713]
  We invoke deep learning (DL) to assist routing in aeronautical ad-hoc networks (AANETs) A deep neural network (DNN) is conceived for mapping the local geographic information observed by the forwarding node into the information required for determining the optimal next hop. We extend the DL-aided routing algorithm to a multi-objective scenario, where we aim for simultaneously minimizing the delay, maximizing the path capacity, and maximizing the path lifetime.
  arXiv  Detail & Related papers  (2021-10-28T14:18:22Z)
• Deep Learning Aided Routing for Space-Air-Ground Integrated Networks Relying on Real Satellite, Flight, and Shipping Data [79.96177511319713]
  Current maritime communications mainly rely on satellites having meager transmission resources, hence suffering from poorer performance than modern terrestrial wireless networks. With the growth of transcontinental air traffic, the promising concept of aeronautical ad hoc networking relying on commercial passenger airplanes is potentially capable of enhancing satellite-based maritime communications via air-to-ground and multi-hop air-to-air links. We propose space-air-ground integrated networks (SAGINs) for supporting ubiquitous maritime communications, where the low-earth-orbit satellite constellations, passenger airplanes, terrestrial base stations, ships, respectively, serve as the space-, air-,
  arXiv  Detail & Related papers  (2021-10-28T14:12:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.