How to Minimize the Weighted Sum AoI in Two-Source Status Update Systems: OMA or NOMA?

• URL: http://arxiv.org/abs/2205.03143v1
• Date: Fri, 6 May 2022 11:18:43 GMT
• Title: How to Minimize the Weighted Sum AoI in Two-Source Status Update Systems: OMA or NOMA?
• Authors: Jixuan Wang and Deli Qiao
• Abstract summary: Two independent sources send update packets to a common destination node in a time-slotted manner under the limit of maximum retransmission rounds. Different multiple access schemes are exploited here over a block-fading multiple access channel (MAC) Online reinforcement learning approaches are proposed to achieve near-optimal age performance.
• Score: 12.041266020039822
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, the minimization of the weighted sum average age of information (AoI) in a two-source status update communication system is studied. Two independent sources send update packets to a common destination node in a time-slotted manner under the limit of maximum retransmission rounds. Different multiple access schemes, i.e., orthogonal multiple access (OMA) and non-orthogonal multiple access (NOMA) are exploited here over a block-fading multiple access channel (MAC). Constrained Markov decision process (CMDP) problems are formulated to describe the AoI minimization problems considering both transmission schemes. The Lagrangian method is utilised to convert CMDP problems to unconstraint Markov decision process (MDP) problems and corresponding algorithms to derive the power allocation policies are obtained. On the other hand, for the case of unknown environments, two online reinforcement learning approaches considering both multiple access schemes are proposed to achieve near-optimal age performance. Numerical simulations validate the improvement of the proposed policy in terms of weighted sum AoI compared to the fixed power transmission policy, and illustrate that NOMA is more favorable in case of larger packet size.

Related papers

• Two-Stage ML-Guided Decision Rules for Sequential Decision Making under Uncertainty [55.06411438416805]
  Sequential Decision Making under Uncertainty (SDMU) is ubiquitous in many domains such as energy, finance, and supply chains. Some SDMU are naturally modeled as Multistage Problems (MSPs) but the resulting optimizations are notoriously challenging from a computational standpoint. This paper introduces a novel approach Two-Stage General Decision Rules (TS-GDR) to generalize the policy space beyond linear functions. The effectiveness of TS-GDR is demonstrated through an instantiation using Deep Recurrent Neural Networks named Two-Stage Deep Decision Rules (TS-LDR)
  arXiv  Detail & Related papers  (2024-05-23T18:19:47Z)
• Deep Reinforcement Learning for Uplink Scheduling in NOMA-URLLC Networks [7.182684187774442]
  This article addresses the problem of Ultra Reliable Low Communications (URLLC) in wireless networks, a framework with particularly stringent constraints imposed by many Internet of Things (IoT) applications from diverse sectors. We propose a novel Deep Reinforcement Learning (DRL) scheduling algorithm, to solve the Non-Orthogonal Multiple Access (NOMA) uplink URLLC scheduling problem involving strict deadlines.
  arXiv  Detail & Related papers  (2023-08-28T12:18:02Z)
• Semi-Infinitely Constrained Markov Decision Processes and Efficient Reinforcement Learning [17.04643707688075]
  We consider a continuum of constraints instead of a finite number of constraints as in the case of ordinary CMDPs. We devise two reinforcement learning algorithms for SICMDPs that we call SI-CRL and SI-CPO. To the best of our knowledge, we are the first to apply tools from semi-infinitely programming (SIP) to solve constrained reinforcement learning problems.
  arXiv  Detail & Related papers  (2023-04-29T12:52:38Z)
• Distributed-Training-and-Execution Multi-Agent Reinforcement Learning for Power Control in HetNet [48.96004919910818]
  We propose a multi-agent deep reinforcement learning (MADRL) based power control scheme for the HetNet. To promote cooperation among agents, we develop a penalty-based Q learning (PQL) algorithm for MADRL systems. In this way, an agent's policy can be learned by other agents more easily, resulting in a more efficient collaboration process.
  arXiv  Detail & Related papers  (2022-12-15T17:01:56Z)
• Age of Semantics in Cooperative Communications: To Expedite Simulation Towards Real via Offline Reinforcement Learning [53.18060442931179]
  We propose the age of semantics (AoS) for measuring semantics freshness of status updates in a cooperative relay communication system. We derive an online deep actor-critic (DAC) learning scheme under the on-policy temporal difference learning framework. We then put forward a novel offline DAC scheme, which estimates the optimal control policy from a previously collected dataset.
  arXiv  Detail & Related papers  (2022-09-19T11:55:28Z)
• MIX-MAB: Reinforcement Learning-based Resource Allocation Algorithm for LoRaWAN [6.22984202194369]
  This paper focuses on improving the resource allocation algorithm in terms of packet delivery ratio (PDR) We propose a resource allocation algorithm that enables the EDs to configure their transmission parameters in a distributed manner. Numerical results show that the proposed solution performs better than the existing schemes in terms of convergence time and PDR.
  arXiv  Detail & Related papers  (2022-06-07T15:50:05Z)
• Processing Network Controls via Deep Reinforcement Learning [0.0]
  dissertation is concerned with theoretical justification and practical application of the advanced policy gradient algorithms. Policy improvement bounds play a crucial role in the theoretical justification of the APG algorithms.
  arXiv  Detail & Related papers  (2022-05-01T04:34:21Z)
• Learning Resilient Radio Resource Management Policies with Graph Neural Networks [124.89036526192268]
  We formulate a resilient radio resource management problem with per-user minimum-capacity constraints. We show that we can parameterize the user selection and power control policies using a finite set of parameters. Thanks to such adaptation, our proposed method achieves a superior tradeoff between the average rate and the 5th percentile rate.
  arXiv  Detail & Related papers  (2022-03-07T19:40:39Z)
• Coded Stochastic ADMM for Decentralized Consensus Optimization with Edge Computing [113.52575069030192]
  Big data, including applications with high security requirements, are often collected and stored on multiple heterogeneous devices, such as mobile devices, drones and vehicles. Due to the limitations of communication costs and security requirements, it is of paramount importance to extract information in a decentralized manner instead of aggregating data to a fusion center. We consider the problem of learning model parameters in a multi-agent system with data locally processed via distributed edge nodes. A class of mini-batch alternating direction method of multipliers (ADMM) algorithms is explored to develop the distributed learning model.
  arXiv  Detail & Related papers  (2020-10-02T10:41:59Z)
• Exploration-Exploitation in Constrained MDPs [79.23623305214275]
  We investigate the exploration-exploitation dilemma in Constrained Markov Decision Processes (CMDPs) While learning in an unknown CMDP, an agent should trade-off exploration to discover new information about the MDP. While the agent will eventually learn a good or optimal policy, we do not want the agent to violate the constraints too often during the learning process.
  arXiv  Detail & Related papers  (2020-03-04T17:03:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.