Multi-Agent Reinforcement Learning Counteracts Delayed CSI in Multi-Satellite Systems

• URL: http://arxiv.org/abs/2603.16470v1
• Date: Tue, 17 Mar 2026 12:58:22 GMT
• Title: Multi-Agent Reinforcement Learning Counteracts Delayed CSI in Multi-Satellite Systems
• Abstract summary: We study the downlink transmission of multiple satellites acting as distributed base stations (BS) to mobile terrestrial users.<n>We propose a multi-agent reinforcement learning (MARL) algorithm which aims for maximising the sum-rate of the users.<n>Specifically, the first stage of DS-PPO maximises the sum-rate for an individual satellite and the second stage maximises the sum-rate when all the satellites cooperate to form a distributed multi-antenna BS.
• Score: 46.11006940623384
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The integration of satellite communication networks with next-generation (NG) technologies is a promising approach towards global connectivity. However, the quality of services is highly dependant on the availability of accurate channel state information (CSI). Channel estimation in satellite communications is challenging due to the high propagation delay between terrestrial users and satellites, which results in outdated CSI observations on the satellite side. In this paper, we study the downlink transmission of multiple satellites acting as distributed base stations (BS) to mobile terrestrial users. We propose a multi-agent reinforcement learning (MARL) algorithm which aims for maximising the sum-rate of the users, while coping with the outdated CSI. We design a novel bi-level optimisation, procedure themes as dual stage proximal policy optimisation (DS-PPO), for tackling the problem of large continuous action spaces as well as of independent and non-identically distributed (non-IID) environments in MARL. Specifically, the first stage of DS-PPO maximises the sum-rate for an individual satellite and the second stage maximises the sum-rate when all the satellites cooperate to form a distributed multi-antenna BS. Our numerical results demonstrate the robustness of DS-PPO to CSI imperfections as well as the sum-rate improvement attached by the use of DS-PPO. In addition, we provide the convergence analysis for the DS-PPO along with the computational complexity.

Related papers

• Robust Design of Integrated Sensing and Communication in LEO Satellite Systems [8.357531008412016]
  This paper investigates integrated sensing and communication (ISAC) in low Earth orbit (LEO) satellite systems.<n>A framework for ISAC in LEO satellite systems is established, where a satellite can simultaneously sense multiple targets and serve multiple communication users (CUs) over the same spectrum.<n>Considering the limited onboard energy of satellite, a novel robust beamforming design algorithm is developed.
  arXiv  Detail & Related papers  (2026-07-14T04:34:55Z)
• Enabling Near-realtime Remote Sensing via Satellite-Ground Collaboration of Large Vision-Language Models [31.075358564392342]
  Large vision-language models (LVLMs) have recently demonstrated great potential in remote sensing (RS) tasks conducted by low Earth orbit (LEO) satellites.<n>We propose Grace, a satellite-ground collaborative system designed for near-realtime LVLM inference in RS tasks.
  arXiv  Detail & Related papers  (2025-10-28T09:48:26Z)
• Joint AoI and Handover Optimization in Space-Air-Ground Integrated Network [48.485907216785904]
  Low Earth orbit (LEO) satellite constellations offer promising solutions with global coverage and reduced latency.<n>Yet struggle with intermittent coverage and intermittent communication windows due to orbital dynamics.<n>Our three-layer design employs hybrid free-space optical (FSO) links for high-capacity satellite-to-ground communication and reliable radio frequency (RF) links for HAP-to-ground transmission.
  arXiv  Detail & Related papers  (2025-09-16T06:16:56Z)
• A Semi-Supervised Federated Learning Framework with Hierarchical Clustering Aggregation for Heterogeneous Satellite Networks [2.5774044809669663]
  Low Earth Orbit (LEO) satellites are emerging as key components of 6G networks.<n>We propose a novel semi-supervised federated learning framework tailored for LEO satellite networks with hierarchical aggregation.<n>Our proposal can significantly reduce processing time (up to 3x) and energy consumption (up to 4x) compared to other comparative methods.
  arXiv  Detail & Related papers  (2025-07-30T02:47:14Z)
• Constellation as a Service: Tailored Connectivity Management in Direct-Satellite-to-Device Networks [51.982277327318656]
  Direct-satellite-to-device (DS2D) communication is emerging as a promising solution for global mobile service extension.<n>The challenge of managing DS2D connectivity for multi-constellations becomes outstanding.<n>This article proposes a Constellation as a Service framework, which treats the entire multi-constellation infrastructure as a shared resource pool.
  arXiv  Detail & Related papers  (2025-07-01T16:06:29Z)
• A Distance Similarity-based Genetic Optimization Algorithm for Satellite Ground Network Planning Considering Feeding Mode [53.71516191515285]
  The low transmission efficiency of the satellite data relay back mission has become a problem that is currently constraining the construction of the system. We propose a distance similarity-based genetic optimization algorithm (DSGA), which considers the state characteristics between the tasks and introduces a weighted Euclidean distance method to determine the similarity between the tasks.
  arXiv  Detail & Related papers  (2024-08-29T06:57:45Z)
• Continual Deep Reinforcement Learning for Decentralized Satellite Routing [34.67962234401005]
  This paper introduces a full solution for decentralized routing in Low Earth Orbit satellite constellations based on continual Deep Reinforcement Learning (DRL) We follow a multi-agent approach, where each satellite acts as an independent decision-making agent. Our solution adapts well to congestion conditions and exploits less loaded paths.
  arXiv  Detail & Related papers  (2024-05-20T18:12:36Z)
• Federated learning for LEO constellations via inter-HAP links [0.0]
  Low Earth Obit (LEO) satellite constellations have seen a sharp increase of deployment in recent years. To apply machine learning (ML) in such applications, the traditional way of downloading satellite data such as imagery to a ground station (GS) is not desirable. We show that existing FL solutions do not fit well in such LEO constellation scenarios because of significant challenges such as excessive convergence delay and unreliable wireless channels.
  arXiv  Detail & Related papers  (2022-05-15T08:22:52Z)
• Learning Emergent Random Access Protocol for LEO Satellite Networks [51.575090080749554]
  We propose a novel grant-free random access solution for LEO SAT networks, dubbed emergent random access channel protocol (eRACH) eRACH is a model-free approach that emerges through interaction with the non-stationary network environment. Compared to RACH, we show from various simulations that our proposed eRACH yields 54.6% higher average network throughput.
  arXiv  Detail & Related papers  (2021-12-03T07:44:45Z)
• Integrating LEO Satellite and UAV Relaying via Reinforcement Learning for Non-Terrestrial Networks [51.05735925326235]
  A mega-constellation of low-earth orbit (LEO) satellites has the potential to enable long-range communication with low latency. We study the problem of forwarding packets between two faraway ground terminals, through an LEO satellite selected from an orbiting constellation. To maximize the end-to-end data rate, the satellite association and HAP location should be optimized. We tackle this problem using deep reinforcement learning (DRL) with a novel action dimension reduction technique.
  arXiv  Detail & Related papers  (2020-05-26T05:39:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.