Multi-UAV Path Planning for Wireless Data Harvesting with Deep
Reinforcement Learning
- URL: http://arxiv.org/abs/2010.12461v3
- Date: Thu, 3 Jun 2021 11:38:05 GMT
- Title: Multi-UAV Path Planning for Wireless Data Harvesting with Deep
Reinforcement Learning
- Authors: Harald Bayerlein, Mirco Theile, Marco Caccamo, David Gesbert
- Abstract summary: We propose a multi-agent reinforcement learning (MARL) approach that can adapt to profound changes in the scenario parameters defining the data harvesting mission.
We show that our proposed network architecture enables the agents to cooperate effectively by carefully dividing the data collection task among themselves.
- Score: 18.266087952180733
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Harvesting data from distributed Internet of Things (IoT) devices with
multiple autonomous unmanned aerial vehicles (UAVs) is a challenging problem
requiring flexible path planning methods. We propose a multi-agent
reinforcement learning (MARL) approach that, in contrast to previous work, can
adapt to profound changes in the scenario parameters defining the data
harvesting mission, such as the number of deployed UAVs, number, position and
data amount of IoT devices, or the maximum flying time, without the need to
perform expensive recomputations or relearn control policies. We formulate the
path planning problem for a cooperative, non-communicating, and homogeneous
team of UAVs tasked with maximizing collected data from distributed IoT sensor
nodes subject to flying time and collision avoidance constraints. The path
planning problem is translated into a decentralized partially observable Markov
decision process (Dec-POMDP), which we solve through a deep reinforcement
learning (DRL) approach, approximating the optimal UAV control policy without
prior knowledge of the challenging wireless channel characteristics in dense
urban environments. By exploiting a combination of centered global and local
map representations of the environment that are fed into convolutional layers
of the agents, we show that our proposed network architecture enables the
agents to cooperate effectively by carefully dividing the data collection task
among themselves, adapt to large complex environments and state spaces, and
make movement decisions that balance data collection goals, flight-time
efficiency, and navigation constraints. Finally, learning a control policy that
generalizes over the scenario parameter space enables us to analyze the
influence of individual parameters on collection performance and provide some
intuition about system-level benefits.
Related papers
- Multi-Agent Reinforcement Learning for Offloading Cellular Communications with Cooperating UAVs [21.195346908715972]
Unmanned aerial vehicles present an alternative means to offload data traffic from terrestrial BSs.
This paper presents a novel approach to efficiently serve multiple UAVs for data offloading from terrestrial BSs.
arXiv Detail & Related papers (2024-02-05T12:36:08Z) - Multi-Objective Optimization for UAV Swarm-Assisted IoT with Virtual
Antenna Arrays [55.736718475856726]
Unmanned aerial vehicle (UAV) network is a promising technology for assisting Internet-of-Things (IoT)
Existing UAV-assisted data harvesting and dissemination schemes require UAVs to frequently fly between the IoTs and access points.
We introduce collaborative beamforming into IoTs and UAVs simultaneously to achieve energy and time-efficient data harvesting and dissemination.
arXiv Detail & Related papers (2023-08-03T02:49:50Z) - Integrated Sensing, Computation, and Communication for UAV-assisted
Federated Edge Learning [52.7230652428711]
Federated edge learning (FEEL) enables privacy-preserving model training through periodic communication between edge devices and the server.
Unmanned Aerial Vehicle (UAV)mounted edge devices are particularly advantageous for FEEL due to their flexibility and mobility in efficient data collection.
arXiv Detail & Related papers (2023-06-05T16:01:33Z) - Muti-Agent Proximal Policy Optimization For Data Freshness in
UAV-assisted Networks [4.042622147977782]
We focus on the case where the collected data is time-sensitive, and it is critical to maintain its timeliness.
Our objective is to optimally design the UAVs' trajectories and the subsets of visited IoT devices such as the global Age-of-Updates (AoU) is minimized.
arXiv Detail & Related papers (2023-03-15T15:03:09Z) - Adaptive Path Planning for UAVs for Multi-Resolution Semantic
Segmentation [28.104584236205405]
A key challenge is planning missions to maximize the value of acquired data in large environments.
This is, for example, relevant for monitoring agricultural fields.
We propose an online planning algorithm which adapts the UAV paths to obtain high-resolution semantic segmentations.
arXiv Detail & Related papers (2022-03-03T11:03:28Z) - Semantic-Aware Collaborative Deep Reinforcement Learning Over Wireless
Cellular Networks [82.02891936174221]
Collaborative deep reinforcement learning (CDRL) algorithms in which multiple agents can coordinate over a wireless network is a promising approach.
In this paper, a novel semantic-aware CDRL method is proposed to enable a group of untrained agents with semantically-linked DRL tasks to collaborate efficiently across a resource-constrained wireless cellular network.
arXiv Detail & Related papers (2021-11-23T18:24:47Z) - Adaptive Anomaly Detection for Internet of Things in Hierarchical Edge
Computing: A Contextual-Bandit Approach [81.5261621619557]
We propose an adaptive anomaly detection scheme with hierarchical edge computing (HEC)
We first construct multiple anomaly detection DNN models with increasing complexity, and associate each of them to a corresponding HEC layer.
Then, we design an adaptive model selection scheme that is formulated as a contextual-bandit problem and solved by using a reinforcement learning policy network.
arXiv Detail & Related papers (2021-08-09T08:45:47Z) - Trajectory Design for UAV-Based Internet-of-Things Data Collection: A
Deep Reinforcement Learning Approach [93.67588414950656]
In this paper, we investigate an unmanned aerial vehicle (UAV)-assisted Internet-of-Things (IoT) system in a 3D environment.
We present a TD3-based trajectory design for completion time minimization (TD3-TDCTM) algorithm.
Our simulation results show the superiority of the proposed TD3-TDCTM algorithm over three conventional non-learning based baseline methods.
arXiv Detail & Related papers (2021-07-23T03:33:29Z) - UAV Path Planning using Global and Local Map Information with Deep
Reinforcement Learning [16.720630804675213]
This work presents a method for autonomous UAV path planning based on deep reinforcement learning (DRL)
We compare coverage path planning ( CPP), where the UAV's goal is to survey an area of interest to data harvesting (DH), where the UAV collects data from distributed Internet of Things (IoT) sensor devices.
By exploiting structured map information of the environment, we train double deep Q-networks (DDQNs) with identical architectures on both distinctly different mission scenarios.
arXiv Detail & Related papers (2020-10-14T09:59:10Z) - UAV Path Planning for Wireless Data Harvesting: A Deep Reinforcement
Learning Approach [18.266087952180733]
We propose a new end-to-end reinforcement learning approach to UAV-enabled data collection from Internet of Things (IoT) devices.
An autonomous drone is tasked with gathering data from distributed sensor nodes subject to limited flying time and obstacle avoidance.
We show that our proposed network architecture enables the agent to make movement decisions for a variety of scenario parameters.
arXiv Detail & Related papers (2020-07-01T15:14:16Z) - Data Freshness and Energy-Efficient UAV Navigation Optimization: A Deep
Reinforcement Learning Approach [88.45509934702913]
We design a navigation policy for multiple unmanned aerial vehicles (UAVs) where mobile base stations (BSs) are deployed.
We incorporate different contextual information such as energy and age of information (AoI) constraints to ensure the data freshness at the ground BS.
By applying the proposed trained model, an effective real-time trajectory policy for the UAV-BSs captures the observable network states over time.
arXiv Detail & Related papers (2020-02-21T07:29:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.