Multi-Agent Reinforcement Learning for Offloading Cellular Communications with Cooperating UAVs
- URL: http://arxiv.org/abs/2402.02957v2
- Date: Fri, 31 May 2024 16:10:28 GMT
- Title: Multi-Agent Reinforcement Learning for Offloading Cellular Communications with Cooperating UAVs
- Authors: Abhishek Mondal, Deepak Mishra, Ganesh Prasad, George C. Alexandropoulos, Azzam Alnahari, Riku Jantti,
- Abstract summary: Unmanned aerial vehicles present an alternative means to offload data traffic from terrestrial BSs.
This paper presents a novel approach to efficiently serve multiple UAVs for data offloading from terrestrial BSs.
- Score: 21.195346908715972
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Effective solutions for intelligent data collection in terrestrial cellular networks are crucial, especially in the context of Internet of Things applications. The limited spectrum and coverage area of terrestrial base stations pose challenges in meeting the escalating data rate demands of network users. Unmanned aerial vehicles, known for their high agility, mobility, and flexibility, present an alternative means to offload data traffic from terrestrial BSs, serving as additional access points. This paper introduces a novel approach to efficiently maximize the utilization of multiple UAVs for data traffic offloading from terrestrial BSs. Specifically, the focus is on maximizing user association with UAVs by jointly optimizing UAV trajectories and users association indicators under quality of service constraints. Since, the formulated UAVs control problem is nonconvex and combinatorial, this study leverages the multi agent reinforcement learning framework. In this framework, each UAV acts as an independent agent, aiming to maintain inter UAV cooperative behavior. The proposed approach utilizes the finite state Markov decision process to account for UAVs velocity constraints and the relationship between their trajectories and state space. A low complexity distributed state action reward state action algorithm is presented to determine UAVs optimal sequential decision making policies over training episodes. The extensive simulation results validate the proposed analysis and offer valuable insights into the optimal UAV trajectories. The derived trajectories demonstrate superior average UAV association performance compared to benchmark techniques such as Q learning and particle swarm optimization.
Related papers
- UAV-enabled Collaborative Beamforming via Multi-Agent Deep Reinforcement Learning [79.16150966434299]
We formulate a UAV-enabled collaborative beamforming multi-objective optimization problem (UCBMOP) to maximize the transmission rate of the UVAA and minimize the energy consumption of all UAVs.
We use the heterogeneous-agent trust region policy optimization (HATRPO) as the basic framework, and then propose an improved HATRPO algorithm, namely HATRPO-UCB.
arXiv Detail & Related papers (2024-04-11T03:19:22Z) - Joint User Association, Interference Cancellation and Power Control for
Multi-IRS Assisted UAV Communications [80.35959154762381]
Intelligent reflecting surface (IRS)-assisted unmanned aerial vehicle (UAV) communications are expected to alleviate the load of ground base stations in a cost-effective way.
Existing studies mainly focus on the deployment and resource allocation of a single IRS instead of multiple IRSs.
We propose a new optimization algorithm for joint IRS-user association, trajectory optimization of UAVs, successive interference cancellation (SIC) decoding order scheduling and power allocation.
arXiv Detail & Related papers (2023-12-08T01:57:10Z) - Integrated Sensing, Computation, and Communication for UAV-assisted
Federated Edge Learning [52.7230652428711]
Federated edge learning (FEEL) enables privacy-preserving model training through periodic communication between edge devices and the server.
Unmanned Aerial Vehicle (UAV)mounted edge devices are particularly advantageous for FEEL due to their flexibility and mobility in efficient data collection.
arXiv Detail & Related papers (2023-06-05T16:01:33Z) - Joint Optimization of Deployment and Trajectory in UAV and IRS-Assisted
IoT Data Collection System [25.32139119893323]
Unmanned aerial vehicles (UAVs) can be applied in many Internet of Things (IoT) systems.
The UAV-IoT wireless channels may be occasionally blocked by trees or high-rise buildings.
This article aims to minimize the energy consumption of the system by jointly optimizing the deployment and trajectory of the UAV.
arXiv Detail & Related papers (2022-10-27T06:27:40Z) - 3D UAV Trajectory and Data Collection Optimisation via Deep
Reinforcement Learning [75.78929539923749]
Unmanned aerial vehicles (UAVs) are now beginning to be deployed for enhancing the network performance and coverage in wireless communication.
It is challenging to obtain an optimal resource allocation scheme for the UAV-assisted Internet of Things (IoT)
In this paper, we design a new UAV-assisted IoT systems relying on the shortest flight path of the UAVs while maximising the amount of data collected from IoT devices.
arXiv Detail & Related papers (2021-06-06T14:08:41Z) - Distributed CNN Inference on Resource-Constrained UAVs for Surveillance
Systems: Design and Optimization [43.9909417652678]
Unmanned Aerial Vehicles (UAVs) have attracted great interest in the last few years owing to their ability to cover large areas and access difficult and hazardous target zones.
Thanks to the advancements in computer vision and machine learning, UAVs are being adopted for a broad range of solutions and applications.
Deep Neural Networks (DNNs) are progressing toward deeper and complex models that prevent them from being executed on-board.
arXiv Detail & Related papers (2021-05-23T20:19:43Z) - Learning-Based UAV Trajectory Optimization with Collision Avoidance and
Connectivity Constraints [0.0]
Unmanned aerial vehicles (UAVs) are expected to be an integral part of wireless networks.
In this paper, we reformulate the multi-UAV trajectory optimization problem with collision avoidance and wireless connectivity constraints.
We propose a decentralized deep reinforcement learning approach to solve the problem.
arXiv Detail & Related papers (2021-04-03T22:22:20Z) - Privacy-Preserving Federated Learning for UAV-Enabled Networks:
Learning-Based Joint Scheduling and Resource Management [45.15174235000158]
Unmanned aerial vehicles (UAVs) are capable of serving as flying base stations (BSs) for supporting data collection, artificial intelligence (AI) model training, and wireless communications.
It is impractical to send raw data of devices to UAV servers for model training.
In this paper, we develop an asynchronous federated learning framework for multi-UAV-enabled networks.
arXiv Detail & Related papers (2020-11-28T18:58:34Z) - Multi-Agent Deep Reinforcement Learning Based Trajectory Planning for
Multi-UAV Assisted Mobile Edge Computing [99.27205900403578]
An unmanned aerial vehicle (UAV)-aided mobile edge computing (MEC) framework is proposed.
We aim to jointly optimize the geographical fairness among all the user equipments (UEs) and the fairness of each UAV's UE-load.
We show that our proposed solution has considerable performance over other traditional algorithms.
arXiv Detail & Related papers (2020-09-23T17:44:07Z) - Data Freshness and Energy-Efficient UAV Navigation Optimization: A Deep
Reinforcement Learning Approach [88.45509934702913]
We design a navigation policy for multiple unmanned aerial vehicles (UAVs) where mobile base stations (BSs) are deployed.
We incorporate different contextual information such as energy and age of information (AoI) constraints to ensure the data freshness at the ground BS.
By applying the proposed trained model, an effective real-time trajectory policy for the UAV-BSs captures the observable network states over time.
arXiv Detail & Related papers (2020-02-21T07:29:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.