Constrained Deep Reinforcement Learning for Energy Sustainable Multi-UAV
based Random Access IoT Networks with NOMA
- URL: http://arxiv.org/abs/2002.00073v2
- Date: Wed, 5 Feb 2020 02:03:28 GMT
- Title: Constrained Deep Reinforcement Learning for Energy Sustainable Multi-UAV
based Random Access IoT Networks with NOMA
- Authors: Sami Khairy, Prasanna Balaprakash, Lin X. Cai, Yu Cheng
- Abstract summary: We apply the Non-Orthogonal Multiple Access technique to improve massive channel access of a wireless IoT network where solar-powered Unmanned Aerial Vehicles (UAVs) relay data from IoT devices to remote servers.
IoT devices contend for accessing the shared wireless channel using an adaptive $p$-persistent slotted Aloha protocol; and the solar-powered UAVs adopt Successive Interference Cancellation (SIC) to decode multiple received data from IoT devices to improve access efficiency.
- Score: 20.160827428161898
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this paper, we apply the Non-Orthogonal Multiple Access (NOMA) technique
to improve the massive channel access of a wireless IoT network where
solar-powered Unmanned Aerial Vehicles (UAVs) relay data from IoT devices to
remote servers. Specifically, IoT devices contend for accessing the shared
wireless channel using an adaptive $p$-persistent slotted Aloha protocol; and
the solar-powered UAVs adopt Successive Interference Cancellation (SIC) to
decode multiple received data from IoT devices to improve access efficiency. To
enable an energy-sustainable capacity-optimal network, we study the joint
problem of dynamic multi-UAV altitude control and multi-cell wireless channel
access management of IoT devices as a stochastic control problem with multiple
energy constraints. To learn an optimal control policy, we first formulate this
problem as a Constrained Markov Decision Process (CMDP), and propose an online
model-free Constrained Deep Reinforcement Learning (CDRL) algorithm based on
Lagrangian primal-dual policy optimization to solve the CMDP. Extensive
simulations demonstrate that our proposed algorithm learns a cooperative policy
among UAVs in which the altitude of UAVs and channel access probability of IoT
devices are dynamically and jointly controlled to attain the maximal long-term
network capacity while maintaining energy sustainability of UAVs. The proposed
algorithm outperforms Deep RL based solutions with reward shaping to account
for energy costs, and achieves a temporal average system capacity which is
$82.4\%$ higher than that of a feasible DRL based solution, and only $6.47\%$
lower compared to that of the energy-constraint-free system.
Related papers
- Multi-Objective Optimization for UAV Swarm-Assisted IoT with Virtual
Antenna Arrays [55.736718475856726]
Unmanned aerial vehicle (UAV) network is a promising technology for assisting Internet-of-Things (IoT)
Existing UAV-assisted data harvesting and dissemination schemes require UAVs to frequently fly between the IoTs and access points.
We introduce collaborative beamforming into IoTs and UAVs simultaneously to achieve energy and time-efficient data harvesting and dissemination.
arXiv Detail & Related papers (2023-08-03T02:49:50Z) - Energy-Efficient Cellular-Connected UAV Swarm Control Optimization [25.299881367750487]
We propose a two-phase command and control (C&C) transmission scheme in a cellular-connected UAV swarm network.
We formulate the problem as a Constrained Markov Decision Process to find the optimal policy.
Our algorithm could maximize the number of UAVs that successfully receive the common C&C under energy constraints.
arXiv Detail & Related papers (2023-03-18T11:42:04Z) - Sparsity-Aware Intelligent Massive Random Access Control in Open RAN: A
Reinforcement Learning Based Approach [61.74489383629319]
Massive random access of devices in the emerging Open Radio Access Network (O-RAN) brings great challenge to the access control and management.
reinforcement-learning (RL)-assisted scheme of closed-loop access control is proposed to preserve sparsity of access requests.
Deep-RL-assisted SAUD is proposed to resolve highly complex environments with continuous and high-dimensional state and action spaces.
arXiv Detail & Related papers (2023-03-05T12:25:49Z) - Optimization for Master-UAV-powered Auxiliary-Aerial-IRS-assisted IoT
Networks: An Option-based Multi-agent Hierarchical Deep Reinforcement
Learning Approach [56.84948632954274]
This paper investigates a master unmanned aerial vehicle (MUAV)-powered Internet of Things (IoT) network.
We propose using a rechargeable auxiliary UAV (AUAV) equipped with an intelligent reflecting surface (IRS) to enhance the communication signals from the MUAV.
Under the proposed model, we investigate the optimal collaboration strategy of these energy-limited UAVs to maximize the accumulated throughput of the IoT network.
arXiv Detail & Related papers (2021-12-20T15:45:28Z) - Deep Reinforcement Learning Based Multidimensional Resource Management
for Energy Harvesting Cognitive NOMA Communications [64.1076645382049]
Combination of energy harvesting (EH), cognitive radio (CR), and non-orthogonal multiple access (NOMA) is a promising solution to improve energy efficiency.
In this paper, we study the spectrum, energy, and time resource management for deterministic-CR-NOMA IoT systems.
arXiv Detail & Related papers (2021-09-17T08:55:48Z) - RIS-assisted UAV Communications for IoT with Wireless Power Transfer
Using Deep Reinforcement Learning [75.677197535939]
We propose a simultaneous wireless power transfer and information transmission scheme for IoT devices with support from unmanned aerial vehicle (UAV) communications.
In a first phase, IoT devices harvest energy from the UAV through wireless power transfer; and then in a second phase, the UAV collects data from the IoT devices through information transmission.
We formulate a Markov decision process and propose two deep reinforcement learning algorithms to solve the optimization problem of maximizing the total network sum-rate.
arXiv Detail & Related papers (2021-08-05T23:55:44Z) - Energy-aware placement optimization of UAV base stations via
decentralized multi-agent Q-learning [3.502112118170715]
Unmanned aerial vehicles serving as aerial base stations (UAV-BSs) can be deployed to provide wireless connectivity to ground devices in events of increased network demand, points-of-failure in existing infrastructure, or disasters.
It is challenging to conserve the energy of UAVs during prolonged coverage tasks, considering their limited on-board battery capacity.
We propose a decentralized Q-learning approach, where each UAV-BS is equipped with an autonomous agent that maximizes the connectivity to ground devices while improving its energy utilization.
arXiv Detail & Related papers (2021-06-01T22:49:42Z) - Data-Driven Random Access Optimization in Multi-Cell IoT Networks with
NOMA [78.60275748518589]
Non-orthogonal multiple access (NOMA) is a key technology to enable massive machine type communications (mMTC) in 5G networks and beyond.
In this paper, NOMA is applied to improve the random access efficiency in high-density spatially-distributed multi-cell wireless IoT networks.
A novel formulation of random channel access management is proposed, in which the transmission probability of each IoT device is tuned to maximize the geometric mean of users' expected capacity.
arXiv Detail & Related papers (2021-01-02T15:21:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.