A Reinforcement Learning Approach to Age of Information in Multi-User Networks with HARQ

• URL: http://arxiv.org/abs/2102.09774v1
• Date: Fri, 19 Feb 2021 07:30:44 GMT
• Title: A Reinforcement Learning Approach to Age of Information in Multi-User Networks with HARQ
• Authors: Elif Tugce Ceran, Deniz Gunduz and Andras Gyorgy
• Abstract summary: Scheduling the transmission of time-sensitive information from a source node to multiple users over error-prone communication channels is studied. Long-term average resource constraint is imposed on the source, which limits the average number of transmissions.
• Score: 1.5469452301122177
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scheduling the transmission of time-sensitive information from a source node to multiple users over error-prone communication channels is studied with the goal of minimizing the long-term average age of information (AoI) at the users. A long-term average resource constraint is imposed on the source, which limits the average number of transmissions. The source can transmit only to a single user at each time slot, and after each transmission, it receives an instantaneous ACK/NACK feedback from the intended receiver, and decides when and to which user to transmit the next update. Assuming the channel statistics are known, the optimal scheduling policy is studied for both the standard automatic repeat request (ARQ) and hybrid ARQ (HARQ) protocols. Then, a reinforcement learning(RL) approach is introduced to find a near-optimal policy, which does not assume any a priori information on the random processes governing the channel states. Different RL methods including average-cost SARSAwith linear function approximation (LFA), upper confidence reinforcement learning (UCRL2), and deep Q-network (DQN) are applied and compared through numerical simulations

Related papers

• Age-of-Gradient Updates for Federated Learning over Random Access Channels [13.337006106442738]
  We study the problem of federated training of a deep neural network (DNN) over a random access channel (RACH) The RACH-FL setting crucially addresses the problem of jointly designing a (i) client selection and (ii) gradient compression strategy. We propose a policy, which we term the ''age-of-gradient'' (AoG) policy in which (i) gradient sparsification is performed using top-K sparsification, (ii) the error correction is performed using memory accumulation, and (iii) the slot transmission probability is obtained by comparing the current local memory
  arXiv  Detail & Related papers  (2024-10-15T18:49:58Z)
• Actor-Critic Scheduling for Path-Aware Air-to-Ground Multipath Multimedia Delivery [5.01187288554981]
  We present a novel scheduler for real-time multimedia delivery in multipath systems based on an Actor-Critic (AC) RL algorithm. The scheduler acting as an RL agent learns in real-time the optimal policy for path selection, path rate allocation and redundancy estimation for flow protection.
  arXiv  Detail & Related papers  (2022-04-28T08:28:25Z)
• Learning Resilient Radio Resource Management Policies with Graph Neural Networks [124.89036526192268]
  We formulate a resilient radio resource management problem with per-user minimum-capacity constraints. We show that we can parameterize the user selection and power control policies using a finite set of parameters. Thanks to such adaptation, our proposed method achieves a superior tradeoff between the average rate and the 5th percentile rate.
  arXiv  Detail & Related papers  (2022-03-07T19:40:39Z)
• Text Generation with Efficient (Soft) Q-Learning [91.47743595382758]
  Reinforcement learning (RL) offers a more flexible solution by allowing users to plug in arbitrary task metrics as reward. We introduce a new RL formulation for text generation from the soft Q-learning perspective. We apply the approach to a wide range of tasks, including learning from noisy/negative examples, adversarial attacks, and prompt generation.
  arXiv  Detail & Related papers  (2021-06-14T18:48:40Z)
• Dynamic Multichannel Access via Multi-agent Reinforcement Learning: Throughput and Fairness Guarantees [9.615742794292943]
  We propose a distributed multichannel access protocol based on multi-agent reinforcement learning (RL) Unlike the previous approaches adjusting channel access probabilities at each time slot, the proposed RL algorithm deterministically selects a set of channel access policies for several consecutive time slots. We perform extensive simulations on realistic traffic environments and demonstrate that the proposed online learning improves both throughput and fairness.
  arXiv  Detail & Related papers  (2021-05-10T02:32:57Z)
• Better than the Best: Gradient-based Improper Reinforcement Learning for Network Scheduling [60.48359567964899]
  We consider the problem of scheduling in constrained queueing networks with a view to minimizing packet delay. We use a policy gradient based reinforcement learning algorithm that produces a scheduler that performs better than the available atomic policies.
  arXiv  Detail & Related papers  (2021-05-01T10:18:34Z)
• Deep Reinforcement Learning for Resource Constrained Multiclass Scheduling in Wireless Networks [0.0]
  In our setup, the available limited bandwidth resources are allocated in order to serve randomly arriving service demands. We propose a distributional Deep Deterministic Policy Gradient (DDPG) algorithm combined with Deep Sets to tackle the problem. Our proposed algorithm is tested on both synthetic and real data, showing consistent gains against state-of-the-art conventional methods.
  arXiv  Detail & Related papers  (2020-11-27T09:49:38Z)
• FedRec: Federated Learning of Universal Receivers over Fading Channels [92.15358738530037]
  We propose a neural network-based symbol detection technique for downlink fading channels. Multiple users collaborate to jointly learn a universal data-driven detector, hence the name FedRec. The performance of the resulting receiver is shown to approach the MAP performance in diverse channel conditions without requiring knowledge of the fading statistics.
  arXiv  Detail & Related papers  (2020-11-14T11:29:55Z)
• Distributional Reinforcement Learning for mmWave Communications with Intelligent Reflectors on a UAV [119.97450366894718]
  A novel communication framework that uses an unmanned aerial vehicle (UAV)-carried intelligent reflector (IR) is proposed. In order to maximize the downlink sum-rate, the optimal precoding matrix (at the base station) and reflection coefficient (at the IR) are jointly derived.
  arXiv  Detail & Related papers  (2020-11-03T16:50:37Z)
• Scheduling Policy and Power Allocation for Federated Learning in NOMA Based MEC [21.267954799102874]
  Federated learning (FL) is a highly pursued machine learning technique that can train a model centrally while keeping data distributed. We propose a new scheduling policy and power allocation scheme using non-orthogonal multiple access (NOMA) settings to maximize the weighted sum data rate. Simulation results show that the proposed scheduling and power allocation scheme can help achieve a higher FL testing accuracy in NOMA based wireless networks.
  arXiv  Detail & Related papers  (2020-06-21T23:07:41Z)
• Millimeter Wave Communications with an Intelligent Reflector: Performance Optimization and Distributional Reinforcement Learning [119.97450366894718]
  A novel framework is proposed to optimize the downlink multi-user communication of a millimeter wave base station. A channel estimation approach is developed to measure the channel state information (CSI) in real-time. A distributional reinforcement learning (DRL) approach is proposed to learn the optimal IR reflection and maximize the expectation of downlink capacity.
  arXiv  Detail & Related papers  (2020-02-24T22:18:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.