Responsive Regulation of Dynamic UAV Communication Networks Based on
Deep Reinforcement Learning
- URL: http://arxiv.org/abs/2108.11012v1
- Date: Wed, 25 Aug 2021 02:04:13 GMT
- Title: Responsive Regulation of Dynamic UAV Communication Networks Based on
Deep Reinforcement Learning
- Authors: Ran Zhang, Duc Minh (Aaron) Nguyen, Miao Wang, Lin X. Cai and Xuemin
(Sherman) Shen
- Abstract summary: We develop an optimal UAV control policy which is capable of identifying the upcoming change in the UAV lineup and relocating the UAVs ahead of the change.
Specifically, a deep reinforcement learning (DRL)-based UAV control framework is developed to maximize the accumulated user satisfaction (US) score for a given time horizon.
In addition, to handle the continuous state and action space, deep deterministic policy gradient (DDPG) algorithm, which is an actor-critic based DRL, is exploited.
- Score: 16.78151396672782
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this chapter, the regulation of Unmanned Aerial Vehicle (UAV)
communication network is investigated in the presence of dynamic changes in the
UAV lineup and user distribution. We target an optimal UAV control policy which
is capable of identifying the upcoming change in the UAV lineup (quit or
join-in) or user distribution, and proactively relocating the UAVs ahead of the
change rather than passively dispatching the UAVs after the change.
Specifically, a deep reinforcement learning (DRL)-based UAV control framework
is developed to maximize the accumulated user satisfaction (US) score for a
given time horizon which is able to handle the change in both the UAV lineup
and user distribution. The framework accommodates the changed dimension of the
state-action space before and after the UAV lineup change by deliberate state
transition design. In addition, to handle the continuous state and action
space, deep deterministic policy gradient (DDPG) algorithm, which is an
actor-critic based DRL, is exploited. Furthermore, to promote the learning
exploration around the timing of the change, the original DDPG is adapted into
an asynchronous parallel computing (APC) structure which leads to a better
training performance in both the critic and actor networks. Finally, extensive
simulations are conducted to validate the convergence of the proposed learning
approach, and demonstrate its capability in jointly handling the dynamics in
UAV lineup and user distribution as well as its superiority over a passive
reaction method.
Related papers
- Cooperative Cognitive Dynamic System in UAV Swarms: Reconfigurable Mechanism and Framework [80.39138462246034]
We propose the cooperative cognitive dynamic system (CCDS) to optimize the management for UAV swarms.
CCDS is a hierarchical and cooperative control structure that enables real-time data processing and decision.
In addition, CCDS can be integrated with the biomimetic mechanism to efficiently allocate tasks for UAV swarms.
arXiv Detail & Related papers (2024-05-18T12:45:00Z) - UAV-enabled Collaborative Beamforming via Multi-Agent Deep Reinforcement Learning [79.16150966434299]
We formulate a UAV-enabled collaborative beamforming multi-objective optimization problem (UCBMOP) to maximize the transmission rate of the UVAA and minimize the energy consumption of all UAVs.
We use the heterogeneous-agent trust region policy optimization (HATRPO) as the basic framework, and then propose an improved HATRPO algorithm, namely HATRPO-UCB.
arXiv Detail & Related papers (2024-04-11T03:19:22Z) - Multi-Agent Reinforcement Learning for Offloading Cellular Communications with Cooperating UAVs [21.195346908715972]
Unmanned aerial vehicles present an alternative means to offload data traffic from terrestrial BSs.
This paper presents a novel approach to efficiently serve multiple UAVs for data offloading from terrestrial BSs.
arXiv Detail & Related papers (2024-02-05T12:36:08Z) - Integrated Sensing, Computation, and Communication for UAV-assisted
Federated Edge Learning [52.7230652428711]
Federated edge learning (FEEL) enables privacy-preserving model training through periodic communication between edge devices and the server.
Unmanned Aerial Vehicle (UAV)mounted edge devices are particularly advantageous for FEEL due to their flexibility and mobility in efficient data collection.
arXiv Detail & Related papers (2023-06-05T16:01:33Z) - Optimization for Master-UAV-powered Auxiliary-Aerial-IRS-assisted IoT
Networks: An Option-based Multi-agent Hierarchical Deep Reinforcement
Learning Approach [56.84948632954274]
This paper investigates a master unmanned aerial vehicle (MUAV)-powered Internet of Things (IoT) network.
We propose using a rechargeable auxiliary UAV (AUAV) equipped with an intelligent reflecting surface (IRS) to enhance the communication signals from the MUAV.
Under the proposed model, we investigate the optimal collaboration strategy of these energy-limited UAVs to maximize the accumulated throughput of the IoT network.
arXiv Detail & Related papers (2021-12-20T15:45:28Z) - Federated Learning for UAV Swarms Under Class Imbalance and Power
Consumption Constraints [6.995852507959362]
It is imperative to investigate the performance of UAV utilization while considering their design limitations.
This paper investigates the deployment of UAV swarms when each UAV carries a machine learning classification task.
arXiv Detail & Related papers (2021-08-23T16:10:14Z) - 3D UAV Trajectory and Data Collection Optimisation via Deep
Reinforcement Learning [75.78929539923749]
Unmanned aerial vehicles (UAVs) are now beginning to be deployed for enhancing the network performance and coverage in wireless communication.
It is challenging to obtain an optimal resource allocation scheme for the UAV-assisted Internet of Things (IoT)
In this paper, we design a new UAV-assisted IoT systems relying on the shortest flight path of the UAVs while maximising the amount of data collected from IoT devices.
arXiv Detail & Related papers (2021-06-06T14:08:41Z) - SREC: Proactive Self-Remedy of Energy-Constrained UAV-Based Networks via
Deep Reinforcement Learning [11.065500588538997]
Energy-aware control for multiple unmanned aerial vehicles (UAVs) is one of the major research interests in UAV based networking.
We study proactive self-remedy of energy-constrained UAV networks when one or more UAVs are short of energy and about to quit for charging.
We propose an energy-aware optimal UAV control policy which proactively relocates the UAVs when any UAV is about to quit the network.
arXiv Detail & Related papers (2020-09-17T20:51:17Z) - UAV Path Planning for Wireless Data Harvesting: A Deep Reinforcement
Learning Approach [18.266087952180733]
We propose a new end-to-end reinforcement learning approach to UAV-enabled data collection from Internet of Things (IoT) devices.
An autonomous drone is tasked with gathering data from distributed sensor nodes subject to limited flying time and obstacle avoidance.
We show that our proposed network architecture enables the agent to make movement decisions for a variety of scenario parameters.
arXiv Detail & Related papers (2020-07-01T15:14:16Z) - Autonomous UAV Navigation: A DDPG-based Deep Reinforcement Learning
Approach [1.552282932199974]
We propose an autonomous UAV path planning framework using deep reinforcement learning approach.
The objective is to employ a self-trained UAV as a flying mobile unit to reach spatially distributed moving or static targets.
arXiv Detail & Related papers (2020-03-21T19:33:00Z) - Federated Learning in the Sky: Joint Power Allocation and Scheduling
with UAV Swarms [98.78553146823829]
Unmanned aerial vehicle (UAV) swarms must exploit machine learning (ML) in order to execute various tasks.
In this paper, a novel framework is proposed to implement distributed learning (FL) algorithms within a UAV swarm.
arXiv Detail & Related papers (2020-02-19T14:04:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.