Dynamic Routing for Integrated Satellite-Terrestrial Networks: A Constrained Multi-Agent Reinforcement Learning Approach

• URL: http://arxiv.org/abs/2401.09455v1
• Date: Sat, 23 Dec 2023 03:36:35 GMT
• Title: Dynamic Routing for Integrated Satellite-Terrestrial Networks: A Constrained Multi-Agent Reinforcement Learning Approach
• Authors: Yifeng Lyu, Han Hu, Rongfei Fan, Zhi Liu, Jianping An, Shiwen Mao
• Abstract summary: We study packet routing with ground stations and satellites working jointly to transmit packets. We propose a novel constrained Multi-Agent reinforcement learning (MARL) dynamic routing algorithm named CMADR. Results demonstrate that CMADR reduces the packet delay by a minimum of 21% and 15%, while meeting stringent energy consumption and packet loss rate constraints, outperforming several baseline algorithms.
• Score: 41.714453335170404
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The integrated satellite-terrestrial network (ISTN) system has experienced significant growth, offering seamless communication services in remote areas with limited terrestrial infrastructure. However, designing a routing scheme for ISTN is exceedingly difficult, primarily due to the heightened complexity resulting from the inclusion of additional ground stations, along with the requirement to satisfy various constraints related to satellite service quality. To address these challenges, we study packet routing with ground stations and satellites working jointly to transmit packets, while prioritizing fast communication and meeting energy efficiency and packet loss requirements. Specifically, we formulate the problem of packet routing with constraints as a max-min problem using the Lagrange method. Then we propose a novel constrained Multi-Agent reinforcement learning (MARL) dynamic routing algorithm named CMADR, which efficiently balances objective improvement and constraint satisfaction during the updating of policy and Lagrange multipliers. Finally, we conduct extensive experiments and an ablation study using the OneWeb and Telesat mega-constellations. Results demonstrate that CMADR reduces the packet delay by a minimum of 21% and 15%, while meeting stringent energy consumption and packet loss rate constraints, outperforming several baseline algorithms.

Related papers

• Efficient Entanglement Routing for Satellite-Aerial-Terrestrial Quantum Networks [28.392847313513503]
  Space-aerial-terrestrial quantum networks (SATQNs) are shaping the future of the global-scale quantum Internet. This paper investigates the collaboration among satellite, aerial, and terrestrial quantum networks to efficiently transmit high-fidelity quantum entanglements over long distances.
  arXiv  Detail & Related papers  (2024-09-20T13:57:32Z)
• A Distance Similarity-based Genetic Optimization Algorithm for Satellite Ground Network Planning Considering Feeding Mode [53.71516191515285]
  The low transmission efficiency of the satellite data relay back mission has become a problem that is currently constraining the construction of the system. We propose a distance similarity-based genetic optimization algorithm (DSGA), which considers the state characteristics between the tasks. The results of simulation experiments show that the DSGA can effectively solve the SGNPFM problem.
  arXiv  Detail & Related papers  (2024-08-29T06:57:45Z)
• Hierarchical Learning and Computing over Space-Ground Integrated Networks [40.19542938629252]
  We propose a hierarchical learning and computing framework to provide global aggregation services for locally trained models on ground IoT devices. We formulate a network energy problem for model aggregation, which turns out to be a Directed Steiner Tree (DST) problem. We propose a topologyaware energy-efficient routing (TAEER) algorithm to solve the DST problem by finding a minimum spanning arborescence on a substitute directed graph.
  arXiv  Detail & Related papers  (2024-08-26T09:05:43Z)
• Shaping Rewards, Shaping Routes: On Multi-Agent Deep Q-Networks for Routing in Satellite Constellation Networks [2.774897240515734]
  We investigate the viability of multi-agent deep Q-networks for routing in satellite constellation networks. We focus on reward shaping and quantifying training convergence for joint optimization of latency and load balancing. We propose a novel hybrid solution based on centralized learning and decentralized control.
  arXiv  Detail & Related papers  (2024-08-04T09:53:57Z)
• Satellite Federated Edge Learning: Architecture Design and Convergence Analysis [47.057886812985984]
  This paper introduces a novel FEEL algorithm, named FEDMEGA, tailored to mega-constellation networks. By integrating inter-satellite links (ISL) for intra-orbit model aggregation, the proposed algorithm significantly reduces the usage of low data rate and intermittent GSL. Our proposed method includes a ring all-reduce based intra-orbit aggregation mechanism, coupled with a network flow-based transmission scheme for global model aggregation.
  arXiv  Detail & Related papers  (2024-04-02T11:59:58Z)
• Security-Sensitive Task Offloading in Integrated Satellite-Terrestrial Networks [15.916368067018169]
  We propose the deployment of LEO satellite edge in an integrated satellite-terrestrial networks (ISTN) structure to support textitsecurity-sensitive computing task offloading. We model the task allocation and offloading order problem as a joint optimization problem to minimize task offloading delay, energy consumption, and the number of attacks while satisfying reliability constraints.
  arXiv  Detail & Related papers  (2024-01-20T07:29:55Z)
• Learning Emergent Random Access Protocol for LEO Satellite Networks [51.575090080749554]
  We propose a novel grant-free random access solution for LEO SAT networks, dubbed emergent random access channel protocol (eRACH) eRACH is a model-free approach that emerges through interaction with the non-stationary network environment. Compared to RACH, we show from various simulations that our proposed eRACH yields 54.6% higher average network throughput.
  arXiv  Detail & Related papers  (2021-12-03T07:44:45Z)
• Deep Learning Aided Routing for Space-Air-Ground Integrated Networks Relying on Real Satellite, Flight, and Shipping Data [79.96177511319713]
  Current maritime communications mainly rely on satellites having meager transmission resources, hence suffering from poorer performance than modern terrestrial wireless networks. With the growth of transcontinental air traffic, the promising concept of aeronautical ad hoc networking relying on commercial passenger airplanes is potentially capable of enhancing satellite-based maritime communications via air-to-ground and multi-hop air-to-air links. We propose space-air-ground integrated networks (SAGINs) for supporting ubiquitous maritime communications, where the low-earth-orbit satellite constellations, passenger airplanes, terrestrial base stations, ships, respectively, serve as the space-, air-,
  arXiv  Detail & Related papers  (2021-10-28T14:12:10Z)
• Integrating LEO Satellite and UAV Relaying via Reinforcement Learning for Non-Terrestrial Networks [51.05735925326235]
  A mega-constellation of low-earth orbit (LEO) satellites has the potential to enable long-range communication with low latency. We study the problem of forwarding packets between two faraway ground terminals, through an LEO satellite selected from an orbiting constellation. To maximize the end-to-end data rate, the satellite association and HAP location should be optimized. We tackle this problem using deep reinforcement learning (DRL) with a novel action dimension reduction technique.
  arXiv  Detail & Related papers  (2020-05-26T05:39:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.