Solving reward-collecting problems with UAVs: a comparison of online optimization and Q-learning

• URL: http://arxiv.org/abs/2112.00141v1
• Date: Tue, 30 Nov 2021 22:27:24 GMT
• Title: Solving reward-collecting problems with UAVs: a comparison of online optimization and Q-learning
• Authors: Yixuan Liu and Chrysafis Vogiatzis and Ruriko Yoshida and Erich Morman
• Abstract summary: We study the problem of identifying a short path from a designated start to a goal, while collecting all rewards and avoiding adversaries that move randomly on the grid. We present a comparison of three methods to solve this problem: namely we implement a Deep Q-Learning model, an $varepsilon$-greedy tabular Q-Learning model, and an online optimization framework. Our experiments, designed using simple grid-world environments with random adversaries, showcase how these approaches work and compare them in terms of performance, accuracy, and computational time.
• Score: 2.4251007104039006
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Uncrewed autonomous vehicles (UAVs) have made significant contributions to reconnaissance and surveillance missions in past US military campaigns. As the prevalence of UAVs increases, there has also been improvements in counter-UAV technology that makes it difficult for them to successfully obtain valuable intelligence within an area of interest. Hence, it has become important that modern UAVs can accomplish their missions while maximizing their chances of survival. In this work, we specifically study the problem of identifying a short path from a designated start to a goal, while collecting all rewards and avoiding adversaries that move randomly on the grid. We also provide a possible application of the framework in a military setting, that of autonomous casualty evacuation. We present a comparison of three methods to solve this problem: namely we implement a Deep Q-Learning model, an $\varepsilon$-greedy tabular Q-Learning model, and an online optimization framework. Our computational experiments, designed using simple grid-world environments with random adversaries showcase how these approaches work and compare them in terms of performance, accuracy, and computational time.

Related papers

• Autonomous Decision Making for UAV Cooperative Pursuit-Evasion Game with Reinforcement Learning [50.33447711072726]
  This paper proposes a deep reinforcement learning-based model for decision-making in multi-role UAV cooperative pursuit-evasion game. The proposed method enables autonomous decision-making of the UAVs in pursuit-evasion game scenarios.
  arXiv  Detail & Related papers  (2024-11-05T10:45:30Z)
• Multi-UAV Multi-RIS QoS-Aware Aerial Communication Systems using DRL and PSO [34.951735976771765]
  Unmanned Aerial Vehicles (UAVs) have attracted the attention of researchers in academia and industry for providing wireless services to ground users. limited resources of UAVs can pose challenges for adopting UAVs for such applications. Our system model considers a UAV swarm that navigates an area, providing wireless communication to ground users with RIS support to improve the coverage of the UAVs.
  arXiv  Detail & Related papers  (2024-06-16T17:53:56Z)
• Autonomous Strike UAVs for Counterterrorism Missions: Challenges and Preliminary Solutions [1.03590082373586]
  Unmanned Aircraft Vehicles (UAVs) are becoming a crucial tool in modern warfare. The use of autonomous UAVs to conduct strike missions against highly valuable targets is the focus of this research.
  arXiv  Detail & Related papers  (2024-03-01T22:52:30Z)
• Meta Reinforcement Learning for Strategic IoT Deployments Coverage in Disaster-Response UAV Swarms [5.57865728456594]
  Unmanned Aerial Vehicles (UAVs) have grabbed the attention of researchers in academia and industry for their potential use in critical emergency applications. These applications include providing wireless services to ground users and collecting data from areas affected by disasters. UAVs' limited resources, energy budget, and strict mission completion time have posed challenges in adopting UAVs for these applications.
  arXiv  Detail & Related papers  (2024-01-20T05:05:39Z)
• Tiny Multi-Agent DRL for Twins Migration in UAV Metaverses: A Multi-Leader Multi-Follower Stackelberg Game Approach [57.15309977293297]
  The synergy between Unmanned Aerial Vehicles (UAVs) and metaverses is giving rise to an emerging paradigm named UAV metaverses. We propose a tiny machine learning-based Stackelberg game framework based on pruning techniques for efficient UT migration in UAV metaverses.
  arXiv  Detail & Related papers  (2024-01-18T02:14:13Z)
• Multi-Objective Optimization for UAV Swarm-Assisted IoT with Virtual Antenna Arrays [55.736718475856726]
  Unmanned aerial vehicle (UAV) network is a promising technology for assisting Internet-of-Things (IoT) Existing UAV-assisted data harvesting and dissemination schemes require UAVs to frequently fly between the IoTs and access points. We introduce collaborative beamforming into IoTs and UAVs simultaneously to achieve energy and time-efficient data harvesting and dissemination.
  arXiv  Detail & Related papers  (2023-08-03T02:49:50Z)
• Q-Learning based system for path planning with unmanned aerial vehicles swarms in obstacle environments [38.82157836789187]
  A Reinforcement Learning based system is proposed for solving this problem in environments with obstacles by making use of Q-Learning. The goal of these paths is to ensure complete coverage of an area with fixed obstacles for tasks, like field prospecting. The results are satisfactory, showing that the system obtains solutions in fewer movements the more UAVs there are.
  arXiv  Detail & Related papers  (2023-03-30T18:37:34Z)
• A Multi-UAV System for Exploration and Target Finding in Cluttered and GPS-Denied Environments [68.31522961125589]
  We propose a framework for a team of UAVs to cooperatively explore and find a target in complex GPS-denied environments with obstacles. The team of UAVs autonomously navigates, explores, detects, and finds the target in a cluttered environment with a known map. Results indicate that the proposed multi-UAV system has improvements in terms of time-cost, the proportion of search area surveyed, as well as successful rates for search and rescue missions.
  arXiv  Detail & Related papers  (2021-07-19T12:54:04Z)
• 3D UAV Trajectory and Data Collection Optimisation via Deep Reinforcement Learning [75.78929539923749]
  Unmanned aerial vehicles (UAVs) are now beginning to be deployed for enhancing the network performance and coverage in wireless communication. It is challenging to obtain an optimal resource allocation scheme for the UAV-assisted Internet of Things (IoT) In this paper, we design a new UAV-assisted IoT systems relying on the shortest flight path of the UAVs while maximising the amount of data collected from IoT devices.
  arXiv  Detail & Related papers  (2021-06-06T14:08:41Z)
• Efficient UAV Trajectory-Planning using Economic Reinforcement Learning [65.91405908268662]
  We introduce REPlanner, a novel reinforcement learning algorithm inspired by economic transactions to distribute tasks between UAVs. We formulate the path planning problem as a multi-agent economic game, where agents can cooperate and compete for resources. As the system computes task distributions via UAV cooperation, it is highly resilient to any change in the swarm size.
  arXiv  Detail & Related papers  (2021-03-03T20:54:19Z)
• Federated Learning for Cellular-connected UAVs: Radio Mapping and Path Planning [2.4366811507669124]
  In this paper, we minimize the travel time of the UAVs, ensuring that a probabilistic connectivity constraint is satisfied. Since the UAVs have different missions and fly over different areas, their collected data carry local information on the network's connectivity. In the first step, the UAVs collaboratively build a global model of the outage probability in the environment. In the second step, by using the global model obtained in the first step and rapidly-exploring random trees (RRTs), we propose an algorithm to optimize UAVs' paths.
  arXiv  Detail & Related papers  (2020-08-23T14:55:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.