Collaborative Ground-Space Communications via Evolutionary Multi-objective Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2404.07450v1
• Date: Thu, 11 Apr 2024 03:13:02 GMT
• Title: Collaborative Ground-Space Communications via Evolutionary Multi-objective Deep Reinforcement Learning
• Authors: Jiahui Li, Geng Sun, Qingqing Wu, Dusit Niyato, Jiawen Kang, Abbas Jamalipour, Victor C. M. Leung,
• Abstract summary: We propose a distributed collaborative beamforming (DCB)-based uplink communication paradigm for enabling ground-space direct communications. DCB treats the terminals that are unable to establish efficient direct connections with the low Earth orbit (LEO) satellites as distributed antennas. We propose an evolutionary multi-objective deep reinforcement learning algorithm to obtain the desirable policies.
• Score: 113.48727062141764
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we propose a distributed collaborative beamforming (DCB)-based uplink communication paradigm for enabling ground-space direct communications. Specifically, DCB treats the terminals that are unable to establish efficient direct connections with the low Earth orbit (LEO) satellites as distributed antennas, forming a virtual antenna array to enhance the terminal-to-satellite uplink achievable rates and durations. However, such systems need multiple trade-off policies that variously balance the terminal-satellite uplink achievable rate, energy consumption of terminals, and satellite switching frequency to satisfy the scenario requirement changes. Thus, we perform a multi-objective optimization analysis and formulate a long-term optimization problem. To address availability in different terminal cluster scales, we reformulate this problem into an action space-reduced and universal multi-objective Markov decision process. Then, we propose an evolutionary multi-objective deep reinforcement learning algorithm to obtain the desirable policies, in which the low-value actions are masked to speed up the training process. As such, the applicability of a one-time trained model can cover more changing terminal-satellite uplink scenarios. Simulation results show that the proposed algorithm outmatches various baselines, and draw some useful insights. Specifically, it is found that DCB enables terminals that cannot reach the uplink achievable threshold to achieve efficient direct uplink transmission, which thus reveals that DCB is an effective solution for enabling direct ground-space communications. Moreover, it reveals that the proposed algorithm achieves multiple policies favoring different objectives and achieving near-optimal uplink achievable rates with low switching frequency.

Related papers

• Latency Optimization in LEO Satellite Communications with Hybrid Beam Pattern and Interference Control [20.19239663262141]
  Low Earth orbit (LEO) satellite communication systems offer high-capacity, low-latency services crucial for next-generation applications. The dense configuration of LEO constellations poses challenges in resource allocation optimization and interference management. This paper proposes a novel framework for optimizing the beam scheduling and resource allocation in multi-beam LEO systems.
  arXiv  Detail & Related papers  (2024-11-14T17:18:24Z)
• A Distance Similarity-based Genetic Optimization Algorithm for Satellite Ground Network Planning Considering Feeding Mode [53.71516191515285]
  The low transmission efficiency of the satellite data relay back mission has become a problem that is currently constraining the construction of the system. We propose a distance similarity-based genetic optimization algorithm (DSGA), which considers the state characteristics between the tasks and introduces a weighted Euclidean distance method to determine the similarity between the tasks.
  arXiv  Detail & Related papers  (2024-08-29T06:57:45Z)
• Generalized Multi-Objective Reinforcement Learning with Envelope Updates in URLLC-enabled Vehicular Networks [12.323383132739195]
  We develop a novel multi-objective reinforcement learning framework to jointly optimize wireless network selection and autonomous driving policies. The proposed framework is designed to maximize the traffic flow and minimize collisions by controlling the vehicle's motion dynamics. The proposed policies enable autonomous vehicles to adopt safe driving behaviors with improved connectivity.
  arXiv  Detail & Related papers  (2024-05-18T16:31:32Z)
• UAV-assisted Semantic Communication with Hybrid Action Reinforcement Learning [19.48293218551122]
  We propose a hybrid action reinforcement learning framework to make decisions on semantic model scale, channel allocation, transmission power, and UAV trajectory. Simulation results indicate that the proposed hybrid action reinforcement learning framework can effectively improve the efficiency of uplink semantic data collection.
  arXiv  Detail & Related papers  (2023-08-18T06:30:18Z)
• Joint Optimization of Deployment and Trajectory in UAV and IRS-Assisted IoT Data Collection System [25.32139119893323]
  Unmanned aerial vehicles (UAVs) can be applied in many Internet of Things (IoT) systems. The UAV-IoT wireless channels may be occasionally blocked by trees or high-rise buildings. This article aims to minimize the energy consumption of the system by jointly optimizing the deployment and trajectory of the UAV.
  arXiv  Detail & Related papers  (2022-10-27T06:27:40Z)
• Federated learning for LEO constellations via inter-HAP links [0.0]
  Low Earth Obit (LEO) satellite constellations have seen a sharp increase of deployment in recent years. To apply machine learning (ML) in such applications, the traditional way of downloading satellite data such as imagery to a ground station (GS) is not desirable. We show that existing FL solutions do not fit well in such LEO constellation scenarios because of significant challenges such as excessive convergence delay and unreliable wireless channels.
  arXiv  Detail & Related papers  (2022-05-15T08:22:52Z)
• Path Design and Resource Management for NOMA enhanced Indoor Intelligent Robots [58.980293789967575]
  A communication enabled indoor intelligent robots (IRs) service framework is proposed. Lego modeling method is proposed, which can deterministically describe the indoor layout and channel state. The investigated radio map is invoked as a virtual environment to train the reinforcement learning agent.
  arXiv  Detail & Related papers  (2020-11-23T21:45:01Z)
• Distributional Reinforcement Learning for mmWave Communications with Intelligent Reflectors on a UAV [119.97450366894718]
  A novel communication framework that uses an unmanned aerial vehicle (UAV)-carried intelligent reflector (IR) is proposed. In order to maximize the downlink sum-rate, the optimal precoding matrix (at the base station) and reflection coefficient (at the IR) are jointly derived.
  arXiv  Detail & Related papers  (2020-11-03T16:50:37Z)
• Meta-Reinforcement Learning for Trajectory Design in Wireless UAV Networks [151.65541208130995]
  A drone base station (DBS) is dispatched to provide uplink connectivity to ground users whose demand is dynamic and unpredictable. In this case, the DBS's trajectory must be adaptively adjusted to satisfy the dynamic user access requests. A meta-learning algorithm is proposed in order to adapt the DBS's trajectory when it encounters novel environments.
  arXiv  Detail & Related papers  (2020-05-25T20:43:59Z)
• Optimization-driven Deep Reinforcement Learning for Robust Beamforming in IRS-assisted Wireless Communications [54.610318402371185]
  Intelligent reflecting surface (IRS) is a promising technology to assist downlink information transmissions from a multi-antenna access point (AP) to a receiver. We minimize the AP's transmit power by a joint optimization of the AP's active beamforming and the IRS's passive beamforming. We propose a deep reinforcement learning (DRL) approach that can adapt the beamforming strategies from past experiences.
  arXiv  Detail & Related papers  (2020-05-25T01:42:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.