Model-aided Deep Reinforcement Learning for Sample-efficient UAV Trajectory Design in IoT Networks

• URL: http://arxiv.org/abs/2104.10403v1
• Date: Wed, 21 Apr 2021 08:25:11 GMT
• Title: Model-aided Deep Reinforcement Learning for Sample-efficient UAV Trajectory Design in IoT Networks
• Authors: Omid Esrafilian, Harald Bayerlein, and David Gesbert
• Abstract summary: We propose a model-aided deep Q-learning approach to guide a flight-time restricted UAV on a data harvesting mission. We show that in comparison with standard DRL approaches, the proposed model-aided approach requires at least one order of magnitude less training data samples to reach identical data collection performance.
• Score: 20.303937220315177
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep Reinforcement Learning (DRL) has become a prominent paradigm to design trajectories for autonomous unmanned aerial vehicles (UAV) used as flying access points in the context of cellular or Internet of Things (IoT) connectivity. However, the prohibitively high training data demand severely restricts the applicability of RL-based trajectory planning in real-world missions. We propose a model-aided deep Q-learning approach that, in contrast to previous work, requires a minimum of expensive training data samples and is able to guide a flight-time restricted UAV on a data harvesting mission without prior knowledge of wireless channel characteristics and limited knowledge of wireless node locations. By exploiting some known reference wireless node positions and channel gain measurements, we seek to learn a model of the environment by estimating unknown node positions and learning the wireless channel characteristics. Interaction with the model allows us to train a deep Q-network (DQN) to approximate the optimal UAV control policy. We show that in comparison with standard DRL approaches, the proposed model-aided approach requires at least one order of magnitude less training data samples to reach identical data collection performance, hence offering a first step towards making DRL a viable solution to the problem.

Related papers

• Handover and SINR-Aware Path Optimization in 5G-UAV mmWave Communication using DRL [0.5315148938765306]
  We propose a novel model-free actor-critic deep reinforcement learning (AC-DRL) framework for path optimization in UAV-assisted 5G mmWave wireless networks. We train an AC-RL agent that enables a UAV connected to a gNB to determine the optimal path to a desired destination in the shortest possible time.
  arXiv  Detail & Related papers  (2025-04-03T15:28:04Z)
• Optimal Transport-Guided Source-Free Adaptation for Face Anti-Spoofing [58.56017169759816]
  We introduce a novel method in which the face anti-spoofing model can be adapted by the client itself to a target domain at test time. Specifically, we develop a prototype-based base model and an optimal transport-guided adaptor. In cross-domain and cross-attack settings, compared with recent methods, our method achieves average relative improvements of 19.17% in HTER and 8.58% in AUC.
  arXiv  Detail & Related papers  (2025-03-29T06:10:34Z)
• Deep Learning for Trajectory Data Management and Mining: A Survey and Beyond [58.63558696061679]
  Trajectory computing is crucial in various practical applications such as location services, urban traffic, and public safety. We present a review of development and recent advances in deep learning for trajectory computing (DL4Traj) Notably, we encapsulate recent advancements in Large Language Models (LLMs) that hold potential to augment trajectory computing.
  arXiv  Detail & Related papers  (2024-03-21T05:57:27Z)
• Joint Path planning and Power Allocation of a Cellular-Connected UAV using Apprenticeship Learning via Deep Inverse Reinforcement Learning [7.760962597460447]
  This paper investigates an interference-aware joint path planning and power allocation mechanism for a cellular-connected unmanned aerial vehicle (UAV) in a sparse suburban environment. The UAV aims to maximize its uplink throughput and minimize the level of interference to the ground user equipment (UEs) connected to the neighbor cellular BSs. An apprenticeship learning method is utilized via inverse reinforcement learning (IRL) based on both Q-learning and deep reinforcement learning (DRL)
  arXiv  Detail & Related papers  (2023-06-15T20:50:05Z)
• Muti-Agent Proximal Policy Optimization For Data Freshness in UAV-assisted Networks [4.042622147977782]
  We focus on the case where the collected data is time-sensitive, and it is critical to maintain its timeliness. Our objective is to optimally design the UAVs' trajectories and the subsets of visited IoT devices such as the global Age-of-Updates (AoU) is minimized.
  arXiv  Detail & Related papers  (2023-03-15T15:03:09Z)
• DL-DRL: A double-level deep reinforcement learning approach for large-scale task scheduling of multi-UAV [65.07776277630228]
  We propose a double-level deep reinforcement learning (DL-DRL) approach based on a divide and conquer framework (DCF) Particularly, we design an encoder-decoder structured policy network in our upper-level DRL model to allocate the tasks to different UAVs. We also exploit another attention based policy network in our lower-level DRL model to construct the route for each UAV, with the objective to maximize the number of executed tasks.
  arXiv  Detail & Related papers  (2022-08-04T04:35:53Z)
• Federated Deep Learning Meets Autonomous Vehicle Perception: Design and Verification [168.67190934250868]
  Federated learning empowered connected autonomous vehicle (FLCAV) has been proposed. FLCAV preserves privacy while reducing communication and annotation costs. It is challenging to determine the network resources and road sensor poses for multi-stage training.
  arXiv  Detail & Related papers  (2022-06-03T23:55:45Z)
• Joint Cluster Head Selection and Trajectory Planning in UAV-Aided IoT Networks by Reinforcement Learning with Sequential Model [4.273341750394231]
  We formulate the problem of jointly designing the UAV's trajectory and selecting cluster heads in the Internet-of-Things network. We propose a novel deep reinforcement learning (DRL) with a sequential model strategy that can effectively learn the policy represented by a sequence-to-sequence neural network. Through extensive simulations, the obtained results show that the proposed DRL method can find the UAV's trajectory that requires much less energy consumption.
  arXiv  Detail & Related papers  (2021-12-01T07:59:53Z)
• Trajectory Design for UAV-Based Internet-of-Things Data Collection: A Deep Reinforcement Learning Approach [93.67588414950656]
  In this paper, we investigate an unmanned aerial vehicle (UAV)-assisted Internet-of-Things (IoT) system in a 3D environment. We present a TD3-based trajectory design for completion time minimization (TD3-TDCTM) algorithm. Our simulation results show the superiority of the proposed TD3-TDCTM algorithm over three conventional non-learning based baseline methods.
  arXiv  Detail & Related papers  (2021-07-23T03:33:29Z)
• 3D UAV Trajectory and Data Collection Optimisation via Deep Reinforcement Learning [75.78929539923749]
  Unmanned aerial vehicles (UAVs) are now beginning to be deployed for enhancing the network performance and coverage in wireless communication. It is challenging to obtain an optimal resource allocation scheme for the UAV-assisted Internet of Things (IoT) In this paper, we design a new UAV-assisted IoT systems relying on the shortest flight path of the UAVs while maximising the amount of data collected from IoT devices.
  arXiv  Detail & Related papers  (2021-06-06T14:08:41Z)
• UAV Path Planning for Wireless Data Harvesting: A Deep Reinforcement Learning Approach [18.266087952180733]
  We propose a new end-to-end reinforcement learning approach to UAV-enabled data collection from Internet of Things (IoT) devices. An autonomous drone is tasked with gathering data from distributed sensor nodes subject to limited flying time and obstacle avoidance. We show that our proposed network architecture enables the agent to make movement decisions for a variety of scenario parameters.
  arXiv  Detail & Related papers  (2020-07-01T15:14:16Z)
• Data Freshness and Energy-Efficient UAV Navigation Optimization: A Deep Reinforcement Learning Approach [88.45509934702913]
  We design a navigation policy for multiple unmanned aerial vehicles (UAVs) where mobile base stations (BSs) are deployed. We incorporate different contextual information such as energy and age of information (AoI) constraints to ensure the data freshness at the ground BS. By applying the proposed trained model, an effective real-time trajectory policy for the UAV-BSs captures the observable network states over time.
  arXiv  Detail & Related papers  (2020-02-21T07:29:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.