Graph Attention-based Reinforcement Learning for Trajectory Design and Resource Assignment in Multi-UAV Assisted Communication

• URL: http://arxiv.org/abs/2401.17880v1
• Date: Wed, 31 Jan 2024 14:37:06 GMT
• Title: Graph Attention-based Reinforcement Learning for Trajectory Design and Resource Assignment in Multi-UAV Assisted Communication
• Authors: Zikai Feng, Di Wu, Mengxing Huang, Chau Yuen
• Abstract summary: It is challenging for UAV base stations (UAV BSs) to realize trajectory design and resource assignment in unknown environments. The cooperation and competition between UAV BSs in the communication network leads to a Markov game problem. In this paper, a novel graph-attention multi-agent trust region (GA-MATR) reinforcement learning framework is proposed to solve the multi-UAV assisted communication problem.
• Score: 20.79743323142469
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the multiple unmanned aerial vehicle (UAV)- assisted downlink communication, it is challenging for UAV base stations (UAV BSs) to realize trajectory design and resource assignment in unknown environments. The cooperation and competition between UAV BSs in the communication network leads to a Markov game problem. Multi-agent reinforcement learning is a significant solution for the above decision-making. However, there are still many common issues, such as the instability of the system and low utilization of historical data, that limit its application. In this paper, a novel graph-attention multi-agent trust region (GA-MATR) reinforcement learning framework is proposed to solve the multi-UAV assisted communication problem. Graph recurrent network is introduced to process and analyze complex topology of the communication network, so as to extract useful information and patterns from observational information. The attention mechanism provides additional weighting for conveyed information, so that the critic network can accurately evaluate the value of behavior for UAV BSs. This provides more reliable feedback signals and helps the actor network update the strategy more effectively. Ablation simulations indicate that the proposed approach attains improved convergence over the baselines. UAV BSs learn the optimal communication strategies to achieve their maximum cumulative rewards. Additionally, multi-agent trust region method with monotonic convergence provides an estimated Nash equilibrium for the multi-UAV assisted communication Markov game.

Related papers

• Predictive Covert Communication Against Multi-UAV Surveillance Using Graph Koopman Autoencoder [27.178522837149053]
  Low Probability of Detection (LPD) communication aims to obscure the presence of radio frequency (RF) signals to evade surveillance. accurately predicting future locations of UAVs is essential for enabling real-time LPD communication. We introduce a novel framework termed predictive covert communication, aimed at minimizing detectability in terrestrial ad-hoc networks under multi-UAV surveillance.
  arXiv  Detail & Related papers  (2024-09-25T16:02:45Z)
• Multi-Agent Reinforcement Learning for Offloading Cellular Communications with Cooperating UAVs [21.195346908715972]
  Unmanned aerial vehicles present an alternative means to offload data traffic from terrestrial BSs. This paper presents a novel approach to efficiently serve multiple UAVs for data offloading from terrestrial BSs.
  arXiv  Detail & Related papers  (2024-02-05T12:36:08Z)
• Graph Koopman Autoencoder for Predictive Covert Communication Against UAV Surveillance [29.15836826461713]
  Low Probability of Detection (LPD) communication aims to obscure the very presence of radio frequency (RF) signals. Unmanned Aerial Vehicles (UAVs) can detect RF signals from the ground by hovering over specific areas of interest. We introduce a novel framework that combines graph neural networks (GNN) with Koopman theory to predict the trajectories of multiple fixed-wing UAVs.
  arXiv  Detail & Related papers  (2024-01-23T23:42:55Z)
• Deep Reinforcement Learning for Interference Management in UAV-based 3D Networks: Potentials and Challenges [137.47736805685457]
  We show that interference can still be effectively mitigated even without knowing its channel information. By harnessing interference, the proposed solutions enable the continued growth of civilian UAVs.
  arXiv  Detail & Related papers  (2023-05-11T18:06:46Z)
• Attention-based Reinforcement Learning for Real-Time UAV Semantic Communication [53.46235596543596]
  We study the problem of air-to-ground ultra-reliable and low-latency communication (URLLC) for a moving ground user. We propose a novel multi-agent deep reinforcement learning framework, coined a graph attention exchange network (GAXNet) GAXNet achieves 6.5x lower latency with the target 0.0000001 error rate, compared to a state-of-the-art baseline framework.
  arXiv  Detail & Related papers  (2021-05-22T12:43:25Z)
• Jamming-Resilient Path Planning for Multiple UAVs via Deep Reinforcement Learning [1.2330326247154968]
  Unmanned aerial vehicles (UAVs) are expected to be an integral part of wireless networks. In this paper, we aim to find collision-free paths for multiple cellular-connected UAVs. We propose an offline temporal difference (TD) learning algorithm with online signal-to-interference-plus-noise ratio mapping to solve the problem.
  arXiv  Detail & Related papers  (2021-04-09T16:52:33Z)
• Learning-Based UAV Trajectory Optimization with Collision Avoidance and Connectivity Constraints [0.0]
  Unmanned aerial vehicles (UAVs) are expected to be an integral part of wireless networks. In this paper, we reformulate the multi-UAV trajectory optimization problem with collision avoidance and wireless connectivity constraints. We propose a decentralized deep reinforcement learning approach to solve the problem.
  arXiv  Detail & Related papers  (2021-04-03T22:22:20Z)
• Efficient UAV Trajectory-Planning using Economic Reinforcement Learning [65.91405908268662]
  We introduce REPlanner, a novel reinforcement learning algorithm inspired by economic transactions to distribute tasks between UAVs. We formulate the path planning problem as a multi-agent economic game, where agents can cooperate and compete for resources. As the system computes task distributions via UAV cooperation, it is highly resilient to any change in the swarm size.
  arXiv  Detail & Related papers  (2021-03-03T20:54:19Z)
• Distributed Conditional Generative Adversarial Networks (GANs) for Data-Driven Millimeter Wave Communications in UAV Networks [116.94802388688653]
  A novel framework is proposed to perform data-driven air-to-ground (A2G) channel estimation for millimeter wave (mmWave) communications in an unmanned aerial vehicle (UAV) wireless network. An effective channel estimation approach is developed, allowing each UAV to train a stand-alone channel model via a conditional generative adversarial network (CGAN) along each beamforming direction. A cooperative framework, based on a distributed CGAN architecture, is developed, allowing each UAV to collaboratively learn the mmWave channel distribution.
  arXiv  Detail & Related papers  (2021-02-02T20:56:46Z)
• Data Freshness and Energy-Efficient UAV Navigation Optimization: A Deep Reinforcement Learning Approach [88.45509934702913]
  We design a navigation policy for multiple unmanned aerial vehicles (UAVs) where mobile base stations (BSs) are deployed. We incorporate different contextual information such as energy and age of information (AoI) constraints to ensure the data freshness at the ground BS. By applying the proposed trained model, an effective real-time trajectory policy for the UAV-BSs captures the observable network states over time.
  arXiv  Detail & Related papers  (2020-02-21T07:29:15Z)
• Federated Learning in the Sky: Joint Power Allocation and Scheduling with UAV Swarms [98.78553146823829]
  Unmanned aerial vehicle (UAV) swarms must exploit machine learning (ML) in order to execute various tasks. In this paper, a novel framework is proposed to implement distributed learning (FL) algorithms within a UAV swarm.
  arXiv  Detail & Related papers  (2020-02-19T14:04:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.