Responsive Regulation of Dynamic UAV Communication Networks Based on Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2108.11012v1
• Date: Wed, 25 Aug 2021 02:04:13 GMT
• Title: Responsive Regulation of Dynamic UAV Communication Networks Based on Deep Reinforcement Learning
• Abstract summary: We develop an optimal UAV control policy which is capable of identifying the upcoming change in the UAV lineup and relocating the UAVs ahead of the change. Specifically, a deep reinforcement learning (DRL)-based UAV control framework is developed to maximize the accumulated user satisfaction (US) score for a given time horizon. In addition, to handle the continuous state and action space, deep deterministic policy gradient (DDPG) algorithm, which is an actor-critic based DRL, is exploited.
• Score: 16.78151396672782
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this chapter, the regulation of Unmanned Aerial Vehicle (UAV) communication network is investigated in the presence of dynamic changes in the UAV lineup and user distribution. We target an optimal UAV control policy which is capable of identifying the upcoming change in the UAV lineup (quit or join-in) or user distribution, and proactively relocating the UAVs ahead of the change rather than passively dispatching the UAVs after the change. Specifically, a deep reinforcement learning (DRL)-based UAV control framework is developed to maximize the accumulated user satisfaction (US) score for a given time horizon which is able to handle the change in both the UAV lineup and user distribution. The framework accommodates the changed dimension of the state-action space before and after the UAV lineup change by deliberate state transition design. In addition, to handle the continuous state and action space, deep deterministic policy gradient (DDPG) algorithm, which is an actor-critic based DRL, is exploited. Furthermore, to promote the learning exploration around the timing of the change, the original DDPG is adapted into an asynchronous parallel computing (APC) structure which leads to a better training performance in both the critic and actor networks. Finally, extensive simulations are conducted to validate the convergence of the proposed learning approach, and demonstrate its capability in jointly handling the dynamics in UAV lineup and user distribution as well as its superiority over a passive reaction method.

Related papers

• MA-HEAD-Net: Adaptive Rule-Guided Multi-Agent DRL for AoI Minimization in UAV-Assisted Emergency Networks [71.01350912967922]
  In post-disaster scenarios, unmanned aerial vehicles (UAVs) are critical for establishing emergency communication networks.<n>This paper investigates age of information (AoI) minimization for UAV-assisted emergency communications with heterogeneous emergency services.<n>We develop MA-HEAD-Net, an adaptive rule-guided multi-agent deep reinforcement learning framework.
  arXiv  Detail & Related papers  (2026-08-02T10:05:50Z)
• Digital Twin Enabled Simultaneous Learning and Modeling for UAV-assisted Secure Communications with Eavesdropping Attacks [13.018533875928048]
  This paper focuses on secure communications in UAV-assisted wireless networks.<n>It comprises multiple legitimate UAVs (LE-UAVs) and an intelligent eavesdropping UAV (EA-UAV)<n>We propose a mode-switching scheme that enables LE-UAVs to dynamically switch between the data transmission and jamming modes.
  arXiv  Detail & Related papers  (2026-03-24T03:31:08Z)
• Spatio-Temporal Attention Enhanced Multi-Agent DRL for UAV-Assisted Wireless Networks with Limited Communications [73.26813382994867]
  We propose a delay-tolerant multi-agent deep reinforcement learning algorithm that integrates a delay-penalized reward to encourage information sharing among UAVs.<n>We show that our new approach achieves over 50% reduction in information delay and 75% throughput gain compared to the conventional MADRL.<n>It is also effective in reducing the need for UAVs' information exchange and thus fostering practical deployment of MADRL in UAV-assisted wireless networks.
  arXiv  Detail & Related papers  (2026-03-23T05:42:50Z)
• Embodied AI-Enhanced IoMT Edge Computing: UAV Trajectory Optimization and Task Offloading with Mobility Prediction [16.23179975760884]
  Unmanned aerial vehicles (UAVs) have been widely used in Internet of Medical Things (IoMT) to provide real-time biomedical edge computing service for wireless body area network (WBAN) users.<n>We propose a novel hierarchical multi-scale Transformer-based user trajectory prediction model based on the users' historical trajectory traces captured by the embodied AI agent.<n>A prediction-enhanced deep reinforcement learning (DRL) algorithm that integrates predicted users' mobility information is designed for intelligently optimizing UAV flight trajectory and task offloading decisions.
  arXiv  Detail & Related papers  (2025-12-24T03:06:37Z)
• AdaptFly: Prompt-Guided Adaptation of Foundation Models for Low-Altitude UAV Networks [10.80018338292861]
  Low-altitude Unmanned Aerial Vehicle (UAV) networks rely on robust semantic segmentation as a foundational enabler for distributed sensing-communication-control co-design.<n>We propose AdaptFly, a prompt-guided test-time adaptation framework that adjusts segmentation models without weight updates.<n>Experiments on UAVid and VDD benchmarks, along with real-world UAV deployments under diverse weather conditions, demonstrate that AdaptFly significantly improves segmentation accuracy and robustness.
  arXiv  Detail & Related papers  (2025-11-13T00:20:37Z)
• LLM Meets the Sky: Heuristic Multi-Agent Reinforcement Learning for Secure Heterogeneous UAV Networks [57.27815890269697]
  This work focuses on maximizing the secrecy rate in heterogeneous UAV networks (HetUAVNs) under energy constraints.<n>We introduce a Large Language Model (LLM)-guided multi-agent learning approach.<n>Results show that our method outperforms existing baselines in secrecy and energy efficiency.
  arXiv  Detail & Related papers  (2025-07-23T04:22:57Z)
• Age of Information Minimization in UAV-Enabled Integrated Sensing and Communication Systems [34.92822911897626]
  Unmanned aerial vehicles (UAVs) equipped with integrated sensing and communication (ISAC) capabilities are envisioned to play a pivotal role in future wireless networks.<n>We propose Age Information (AoI) system that simultaneously performs target sensing and multi-user communication.
  arXiv  Detail & Related papers  (2025-07-18T18:17:09Z)
• Autonomous Decision Making for UAV Cooperative Pursuit-Evasion Game with Reinforcement Learning [50.33447711072726]
  This paper proposes a deep reinforcement learning-based model for decision-making in multi-role UAV cooperative pursuit-evasion game. The proposed method enables autonomous decision-making of the UAVs in pursuit-evasion game scenarios.
  arXiv  Detail & Related papers  (2024-11-05T10:45:30Z)
• Cooperative Cognitive Dynamic System in UAV Swarms: Reconfigurable Mechanism and Framework [80.39138462246034]
  We propose the cooperative cognitive dynamic system (CCDS) to optimize the management for UAV swarms. CCDS is a hierarchical and cooperative control structure that enables real-time data processing and decision. In addition, CCDS can be integrated with the biomimetic mechanism to efficiently allocate tasks for UAV swarms.
  arXiv  Detail & Related papers  (2024-05-18T12:45:00Z)
• UAV-enabled Collaborative Beamforming via Multi-Agent Deep Reinforcement Learning [79.16150966434299]
  We formulate a UAV-enabled collaborative beamforming multi-objective optimization problem (UCBMOP) to maximize the transmission rate of the UVAA and minimize the energy consumption of all UAVs. We use the heterogeneous-agent trust region policy optimization (HATRPO) as the basic framework, and then propose an improved HATRPO algorithm, namely HATRPO-UCB.
  arXiv  Detail & Related papers  (2024-04-11T03:19:22Z)
• Multi-Agent Reinforcement Learning for Offloading Cellular Communications with Cooperating UAVs [21.195346908715972]
  Unmanned aerial vehicles present an alternative means to offload data traffic from terrestrial BSs. This paper presents a novel approach to efficiently serve multiple UAVs for data offloading from terrestrial BSs.
  arXiv  Detail & Related papers  (2024-02-05T12:36:08Z)
• Optimization for Master-UAV-powered Auxiliary-Aerial-IRS-assisted IoT Networks: An Option-based Multi-agent Hierarchical Deep Reinforcement Learning Approach [56.84948632954274]
  This paper investigates a master unmanned aerial vehicle (MUAV)-powered Internet of Things (IoT) network. We propose using a rechargeable auxiliary UAV (AUAV) equipped with an intelligent reflecting surface (IRS) to enhance the communication signals from the MUAV. Under the proposed model, we investigate the optimal collaboration strategy of these energy-limited UAVs to maximize the accumulated throughput of the IoT network.
  arXiv  Detail & Related papers  (2021-12-20T15:45:28Z)
• Federated Learning for UAV Swarms Under Class Imbalance and Power Consumption Constraints [6.995852507959362]
  It is imperative to investigate the performance of UAV utilization while considering their design limitations. This paper investigates the deployment of UAV swarms when each UAV carries a machine learning classification task.
  arXiv  Detail & Related papers  (2021-08-23T16:10:14Z)
• 3D UAV Trajectory and Data Collection Optimisation via Deep Reinforcement Learning [75.78929539923749]
  Unmanned aerial vehicles (UAVs) are now beginning to be deployed for enhancing the network performance and coverage in wireless communication. It is challenging to obtain an optimal resource allocation scheme for the UAV-assisted Internet of Things (IoT) In this paper, we design a new UAV-assisted IoT systems relying on the shortest flight path of the UAVs while maximising the amount of data collected from IoT devices.
  arXiv  Detail & Related papers  (2021-06-06T14:08:41Z)
• SREC: Proactive Self-Remedy of Energy-Constrained UAV-Based Networks via Deep Reinforcement Learning [11.065500588538997]
  Energy-aware control for multiple unmanned aerial vehicles (UAVs) is one of the major research interests in UAV based networking. We study proactive self-remedy of energy-constrained UAV networks when one or more UAVs are short of energy and about to quit for charging. We propose an energy-aware optimal UAV control policy which proactively relocates the UAVs when any UAV is about to quit the network.
  arXiv  Detail & Related papers  (2020-09-17T20:51:17Z)
• UAV Path Planning for Wireless Data Harvesting: A Deep Reinforcement Learning Approach [18.266087952180733]
  We propose a new end-to-end reinforcement learning approach to UAV-enabled data collection from Internet of Things (IoT) devices. An autonomous drone is tasked with gathering data from distributed sensor nodes subject to limited flying time and obstacle avoidance. We show that our proposed network architecture enables the agent to make movement decisions for a variety of scenario parameters.
  arXiv  Detail & Related papers  (2020-07-01T15:14:16Z)
• Autonomous UAV Navigation: A DDPG-based Deep Reinforcement Learning Approach [1.552282932199974]
  We propose an autonomous UAV path planning framework using deep reinforcement learning approach. The objective is to employ a self-trained UAV as a flying mobile unit to reach spatially distributed moving or static targets.
  arXiv  Detail & Related papers  (2020-03-21T19:33:00Z)
• Federated Learning in the Sky: Joint Power Allocation and Scheduling with UAV Swarms [98.78553146823829]
  Unmanned aerial vehicle (UAV) swarms must exploit machine learning (ML) in order to execute various tasks. In this paper, a novel framework is proposed to implement distributed learning (FL) algorithms within a UAV swarm.
  arXiv  Detail & Related papers  (2020-02-19T14:04:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.