A Federated Reinforcement Learning Framework for Link Activation in Multi-link Wi-Fi Networks

• URL: http://arxiv.org/abs/2304.14720v1
• Date: Fri, 28 Apr 2023 09:39:10 GMT
• Title: A Federated Reinforcement Learning Framework for Link Activation in Multi-link Wi-Fi Networks
• Authors: Rashid Ali and Boris Bellalta
• Abstract summary: Multi-link operation (MLO) can result in higher interference and channel contention, leading to lower performance and reliability. In this paper, we propose the use of a collaborative machine learning approach to train models across multiple distributed agents without exchanging data. Results show that the FRL-based decentralized MLO-LA strategy achieves a better throughput fairness, and so a higher reliability.
• Score: 3.093231349723552
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Next-generation Wi-Fi networks are looking forward to introducing new features like multi-link operation (MLO) to both achieve higher throughput and lower latency. However, given the limited number of available channels, the use of multiple links by a group of contending Basic Service Sets (BSSs) can result in higher interference and channel contention, thus potentially leading to lower performance and reliability. In such a situation, it could be better for all contending BSSs to use less links if that contributes to reduce channel access contention. Recently, reinforcement learning (RL) has proven its potential for optimizing resource allocation in wireless networks. However, the independent operation of each wireless network makes difficult -- if not almost impossible -- for each individual network to learn a good configuration. To solve this issue, in this paper, we propose the use of a Federated Reinforcement Learning (FRL) framework, i.e., a collaborative machine learning approach to train models across multiple distributed agents without exchanging data, to collaboratively learn the the best MLO-Link Allocation (LA) strategy by a group of neighboring BSSs. The simulation results show that the FRL-based decentralized MLO-LA strategy achieves a better throughput fairness, and so a higher reliability -- because it allows the different BSSs to find a link allocation strategy which maximizes the minimum achieved data rate -- compared to fixed, random and RL-based MLO-LA schemes.

Related papers

• Enhancing Spectrum Efficiency in 6G Satellite Networks: A GAIL-Powered Policy Learning via Asynchronous Federated Inverse Reinforcement Learning [67.95280175998792]
  A novel adversarial imitation learning (GAIL)-powered policy learning approach is proposed for optimizing beamforming, spectrum allocation, and remote user equipment (RUE) association ins. We employ inverse RL (IRL) to automatically learn reward functions without manual tuning. We show that the proposed MA-AL method outperforms traditional RL approaches, achieving a $14.6%$ improvement in convergence and reward value.
  arXiv  Detail & Related papers  (2024-09-27T13:05:02Z)
• AdapShare: An RL-Based Dynamic Spectrum Sharing Solution for O-RAN [1.906179410714637]
  AdapShare is an ORAN-compatible solution leveraging Reinforcement Learning for intent-based spectrum management. By employing RL agents, AdapShare intelligently learns network demand patterns and uses them to allocate resources. AdapShare outperforms a quasi-static resource allocation scheme based on long-term network demand statistics.
  arXiv  Detail & Related papers  (2024-08-29T18:10:36Z)
• A Distributed Neural Linear Thompson Sampling Framework to Achieve URLLC in Industrial IoT [16.167107624956294]
  Industrial Internet of Things (IIoT) networks will provide Ultra-Reliable Low-Latency Communication (URLLC) to support critical processes. Standard protocols for allocating wireless resources may not optimize the latency-reliability trade-off, especially for uplink communication.
  arXiv  Detail & Related papers  (2023-11-21T12:22:04Z)
• RHFedMTL: Resource-Aware Hierarchical Federated Multi-Task Learning [11.329273673732217]
  Federated learning is an effective way to enable AI over massive distributed nodes with security. It is challenging to ensure the privacy while maintain a coupled multi-task learning across multiple base stations (BSs) and terminals. In this paper, inspired by the natural cloud-BS-terminal hierarchy of cellular works, we provide a viable resource-aware hierarchical federated MTL (RHFedMTL) solution.
  arXiv  Detail & Related papers  (2023-06-01T13:49:55Z)
• Decentralized Learning over Wireless Networks: The Effect of Broadcast with Random Access [56.91063444859008]
  We investigate the impact of broadcast transmission and probabilistic random access policy on the convergence performance of D-SGD. Our results demonstrate that optimizing the access probability to maximize the expected number of successful links is a highly effective strategy for accelerating the system convergence.
  arXiv  Detail & Related papers  (2023-05-12T10:32:26Z)
• Intelligent Load Balancing and Resource Allocation in O-RAN: A Multi-Agent Multi-Armed Bandit Approach [4.834203844100679]
  We propose a multi-agent multi-armed bandit for load balancing and resource allocation (mmLBRA) scheme. We also present the mmLBRA-LB and mmLBRA-RA sub-schemes that can operate independently in non-realtime RAN intelligent controller (Non-RT RIC) and near-RT RIC, respectively.
  arXiv  Detail & Related papers  (2023-03-25T04:42:30Z)
• Predictive GAN-powered Multi-Objective Optimization for Hybrid Federated Split Learning [56.125720497163684]
  We propose a hybrid federated split learning framework in wireless networks. We design a parallel computing scheme for model splitting without label sharing, and theoretically analyze the influence of the delayed gradient caused by the scheme on the convergence speed.
  arXiv  Detail & Related papers  (2022-09-02T10:29:56Z)
• Semantic-Aware Collaborative Deep Reinforcement Learning Over Wireless Cellular Networks [82.02891936174221]
  Collaborative deep reinforcement learning (CDRL) algorithms in which multiple agents can coordinate over a wireless network is a promising approach. In this paper, a novel semantic-aware CDRL method is proposed to enable a group of untrained agents with semantically-linked DRL tasks to collaborate efficiently across a resource-constrained wireless cellular network.
  arXiv  Detail & Related papers  (2021-11-23T18:24:47Z)
• Federated Learning over Wireless IoT Networks with Optimized Communication and Resources [98.18365881575805]
  Federated learning (FL) as a paradigm of collaborative learning techniques has obtained increasing research attention. It is of interest to investigate fast responding and accurate FL schemes over wireless systems. We show that the proposed communication-efficient federated learning framework converges at a strong linear rate.
  arXiv  Detail & Related papers  (2021-10-22T13:25:57Z)
• Low-Latency Federated Learning over Wireless Channels with Differential Privacy [142.5983499872664]
  In federated learning (FL), model training is distributed over clients and local models are aggregated by a central server. In this paper, we aim to minimize FL training delay over wireless channels, constrained by overall training performance as well as each client's differential privacy (DP) requirement.
  arXiv  Detail & Related papers  (2021-06-20T13:51:18Z)
• Hierarchical Reinforcement Learning for Relay Selection and Power Optimization in Two-Hop Cooperative Relay Network [7.5377621697101205]
  We study the outage probability minimizing problem subjected to a total transmission power constraint in a two-hop cooperative relay network. We use reinforcement learning (RL) methods to learn strategies for relay selection and power allocation. We propose a hierarchical reinforcement learning (HRL) framework and training algorithm.
  arXiv  Detail & Related papers  (2020-11-10T04:47:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.