Dynamic Routing for Integrated Satellite-Terrestrial Networks: A
Constrained Multi-Agent Reinforcement Learning Approach
- URL: http://arxiv.org/abs/2401.09455v1
- Date: Sat, 23 Dec 2023 03:36:35 GMT
- Title: Dynamic Routing for Integrated Satellite-Terrestrial Networks: A
Constrained Multi-Agent Reinforcement Learning Approach
- Authors: Yifeng Lyu, Han Hu, Rongfei Fan, Zhi Liu, Jianping An, Shiwen Mao
- Abstract summary: We study packet routing with ground stations and satellites working jointly to transmit packets.
We propose a novel constrained Multi-Agent reinforcement learning (MARL) dynamic routing algorithm named CMADR.
Results demonstrate that CMADR reduces the packet delay by a minimum of 21% and 15%, while meeting stringent energy consumption and packet loss rate constraints, outperforming several baseline algorithms.
- Score: 41.714453335170404
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The integrated satellite-terrestrial network (ISTN) system has experienced
significant growth, offering seamless communication services in remote areas
with limited terrestrial infrastructure. However, designing a routing scheme
for ISTN is exceedingly difficult, primarily due to the heightened complexity
resulting from the inclusion of additional ground stations, along with the
requirement to satisfy various constraints related to satellite service
quality. To address these challenges, we study packet routing with ground
stations and satellites working jointly to transmit packets, while prioritizing
fast communication and meeting energy efficiency and packet loss requirements.
Specifically, we formulate the problem of packet routing with constraints as a
max-min problem using the Lagrange method. Then we propose a novel constrained
Multi-Agent reinforcement learning (MARL) dynamic routing algorithm named
CMADR, which efficiently balances objective improvement and constraint
satisfaction during the updating of policy and Lagrange multipliers. Finally,
we conduct extensive experiments and an ablation study using the OneWeb and
Telesat mega-constellations. Results demonstrate that CMADR reduces the packet
delay by a minimum of 21% and 15%, while meeting stringent energy consumption
and packet loss rate constraints, outperforming several baseline algorithms.
Related papers
- Satellite Federated Edge Learning: Architecture Design and Convergence Analysis [47.057886812985984]
This paper introduces a novel FEEL algorithm, named FEDMEGA, tailored to mega-constellation networks.
By integrating inter-satellite links (ISL) for intra-orbit model aggregation, the proposed algorithm significantly reduces the usage of low data rate and intermittent GSL.
Our proposed method includes a ring all-reduce based intra-orbit aggregation mechanism, coupled with a network flow-based transmission scheme for global model aggregation.
arXiv Detail & Related papers (2024-04-02T11:59:58Z) - Security-Sensitive Task Offloading in Integrated Satellite-Terrestrial Networks [15.916368067018169]
We propose the deployment of LEO satellite edge in an integrated satellite-terrestrial networks (ISTN) structure to support textitsecurity-sensitive computing task offloading.
We model the task allocation and offloading order problem as a joint optimization problem to minimize task offloading delay, energy consumption, and the number of attacks while satisfying reliability constraints.
arXiv Detail & Related papers (2024-01-20T07:29:55Z) - Federated learning for LEO constellations via inter-HAP links [0.0]
Low Earth Obit (LEO) satellite constellations have seen a sharp increase of deployment in recent years.
To apply machine learning (ML) in such applications, the traditional way of downloading satellite data such as imagery to a ground station (GS) is not desirable.
We show that existing FL solutions do not fit well in such LEO constellation scenarios because of significant challenges such as excessive convergence delay and unreliable wireless channels.
arXiv Detail & Related papers (2022-05-15T08:22:52Z) - Learning Emergent Random Access Protocol for LEO Satellite Networks [51.575090080749554]
We propose a novel grant-free random access solution for LEO SAT networks, dubbed emergent random access channel protocol (eRACH)
eRACH is a model-free approach that emerges through interaction with the non-stationary network environment.
Compared to RACH, we show from various simulations that our proposed eRACH yields 54.6% higher average network throughput.
arXiv Detail & Related papers (2021-12-03T07:44:45Z) - Deep Learning Aided Routing for Space-Air-Ground Integrated Networks
Relying on Real Satellite, Flight, and Shipping Data [79.96177511319713]
Current maritime communications mainly rely on satellites having meager transmission resources, hence suffering from poorer performance than modern terrestrial wireless networks.
With the growth of transcontinental air traffic, the promising concept of aeronautical ad hoc networking relying on commercial passenger airplanes is potentially capable of enhancing satellite-based maritime communications via air-to-ground and multi-hop air-to-air links.
We propose space-air-ground integrated networks (SAGINs) for supporting ubiquitous maritime communications, where the low-earth-orbit satellite constellations, passenger airplanes, terrestrial base stations, ships, respectively, serve as the space-, air-,
arXiv Detail & Related papers (2021-10-28T14:12:10Z) - Packet Routing with Graph Attention Multi-agent Reinforcement Learning [4.78921052969006]
We develop a model-free and data-driven routing strategy by leveraging reinforcement learning (RL)
Considering the graph nature of the network topology, we design a multi-agent RL framework in combination with Graph Neural Network (GNN)
arXiv Detail & Related papers (2021-07-28T06:20:34Z) - Integrating LEO Satellites and Multi-UAV Reinforcement Learning for
Hybrid FSO/RF Non-Terrestrial Networks [55.776497048509185]
A mega-constellation of low-altitude earth orbit satellites (SATs) and burgeoning unmanned aerial vehicles (UAVs) are promising enablers for high-speed and long-distance communications in beyond fifth-generation (5G) systems.
We investigate the problem of forwarding packets between two faraway ground terminals through SAT and UAV relays using either millimeter-wave (mmWave) radio-frequency (RF) or free-space optical (FSO) link.
arXiv Detail & Related papers (2020-10-20T09:07:10Z) - Integrating LEO Satellite and UAV Relaying via Reinforcement Learning
for Non-Terrestrial Networks [51.05735925326235]
A mega-constellation of low-earth orbit (LEO) satellites has the potential to enable long-range communication with low latency.
We study the problem of forwarding packets between two faraway ground terminals, through an LEO satellite selected from an orbiting constellation.
To maximize the end-to-end data rate, the satellite association and HAP location should be optimized.
We tackle this problem using deep reinforcement learning (DRL) with a novel action dimension reduction technique.
arXiv Detail & Related papers (2020-05-26T05:39:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.