Consensus-based Decentralized Multi-agent Reinforcement Learning for Random Access Network Optimization

• URL: http://arxiv.org/abs/2508.07001v1
• Date: Sat, 09 Aug 2025 14:39:27 GMT
• Title: Consensus-based Decentralized Multi-agent Reinforcement Learning for Random Access Network Optimization
• Authors: Myeung Suk Oh, Zhiyao Zhang, FNU Hairi, Alvaro Velasquez, Jia Liu,
• Abstract summary: We design an effective random access medium access control protocol to minimize collisions and ensure transmission fairness across devices.<n>Our proposed MARL algorithm can significantly improve RA network performance compared to other baselines.
• Score: 10.232557034642015
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With wireless devices increasingly forming a unified smart network for seamless, user-friendly operations, random access (RA) medium access control (MAC) design is considered a key solution for handling unpredictable data traffic from multiple terminals. However, it remains challenging to design an effective RA-based MAC protocol to minimize collisions and ensure transmission fairness across the devices. While existing multi-agent reinforcement learning (MARL) approaches with centralized training and decentralized execution (CTDE) have been proposed to optimize RA performance, their reliance on centralized training and the significant overhead required for information collection can make real-world applications unrealistic. In this work, we adopt a fully decentralized MARL architecture, where policy learning does not rely on centralized tasks but leverages consensus-based information exchanges across devices. We design our MARL algorithm over an actor-critic (AC) network and propose exchanging only local rewards to minimize communication overhead. Furthermore, we provide a theoretical proof of global convergence for our approach. Numerical experiments show that our proposed MARL algorithm can significantly improve RA network performance compared to other baselines.

Related papers

• RESIST: Resilient Decentralized Learning Using Consensus Gradient Descent [11.22833419439317]
  Empirical robustness risk (ERM) is a cornerstone of modern machine learning (ML)<n>This paper focuses on the man-in-the-middle (MITM) attack, which can cause models to deviate significantly from their intended ERM solutions.<n>We propose RESIST, an algorithm designed to be robust against adversarially compromised communication links.
  arXiv  Detail & Related papers  (2025-02-11T21:48:10Z)
• An Offline Multi-Agent Reinforcement Learning Framework for Radio Resource Management [5.771885923067511]
  offline multi-agent reinforcement learning (MARL) addresses key limitations of online MARL.<n>We propose an offline MARL algorithm for radio resource management (RRM)<n>We evaluate three training paradigms: centralized, independent, and centralized training with decentralized execution (CTDE)
  arXiv  Detail & Related papers  (2025-01-22T16:25:46Z)
• Heterogeneous Multi-Agent Reinforcement Learning for Distributed Channel Access in WLANs [47.600901884970845]
  This paper investigates the use of multi-agent reinforcement learning (MARL) to address distributed channel access in wireless local area networks.<n>In particular, we consider the challenging yet more practical case where the agents heterogeneously adopt value-based or policy-based reinforcement learning algorithms to train the model.<n>We propose a heterogeneous MARL training framework, named QPMIX, which adopts a centralized training with distributed execution paradigm to enable heterogeneous agents to collaborate.
  arXiv  Detail & Related papers  (2024-12-18T13:50:31Z)
• Design Optimization of NOMA Aided Multi-STAR-RIS for Indoor Environments: A Convex Approximation Imitated Reinforcement Learning Approach [51.63921041249406]
  Non-orthogonal multiple access (NOMA) enables multiple users to share the same frequency band, and simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) deploying STAR-RIS indoors presents challenges in interference mitigation, power consumption, and real-time configuration. A novel network architecture utilizing multiple access points (APs), STAR-RISs, and NOMA is proposed for indoor communication.
  arXiv  Detail & Related papers  (2024-06-19T07:17:04Z)
• MARLIN: Soft Actor-Critic based Reinforcement Learning for Congestion Control in Real Networks [63.24965775030673]
  We propose a novel Reinforcement Learning (RL) approach to design generic Congestion Control (CC) algorithms. Our solution, MARLIN, uses the Soft Actor-Critic algorithm to maximize both entropy and return. We trained MARLIN on a real network with varying background traffic patterns to overcome the sim-to-real mismatch.
  arXiv  Detail & Related papers  (2023-02-02T18:27:20Z)
• Adaptive Stochastic ADMM for Decentralized Reinforcement Learning in Edge Industrial IoT [106.83952081124195]
  Reinforcement learning (RL) has been widely investigated and shown to be a promising solution for decision-making and optimal control processes. We propose an adaptive ADMM (asI-ADMM) algorithm and apply it to decentralized RL with edge-computing-empowered IIoT networks. Experiment results show that our proposed algorithms outperform the state of the art in terms of communication costs and scalability, and can well adapt to complex IoT environments.
  arXiv  Detail & Related papers  (2021-06-30T16:49:07Z)
• Cooperative Multi-Agent Reinforcement Learning Based Distributed Dynamic Spectrum Access in Cognitive Radio Networks [46.723006378363785]
  Dynamic spectrum access (DSA) is a promising paradigm to remedy the problem of inefficient spectrum utilization. In this paper, we investigate the distributed DSA problem for multi-user in a typical cognitive radio network. We employ the deep recurrent Q-network (DRQN) to address the partial observability of the state for each cognitive user.
  arXiv  Detail & Related papers  (2021-06-17T06:52:21Z)
• Reconfigurable Intelligent Surface Assisted Mobile Edge Computing with Heterogeneous Learning Tasks [53.1636151439562]
  Mobile edge computing (MEC) provides a natural platform for AI applications. We present an infrastructure to perform machine learning tasks at an MEC with the assistance of a reconfigurable intelligent surface (RIS) Specifically, we minimize the learning error of all participating users by jointly optimizing transmit power of mobile users, beamforming vectors of the base station, and the phase-shift matrix of the RIS.
  arXiv  Detail & Related papers  (2020-12-25T07:08:50Z)
• Toward Multiple Federated Learning Services Resource Sharing in Mobile Edge Networks [88.15736037284408]
  We study a new model of multiple federated learning services at the multi-access edge computing server. We propose a joint resource optimization and hyper-learning rate control problem, namely MS-FEDL. Our simulation results demonstrate the convergence performance of our proposed algorithms.
  arXiv  Detail & Related papers  (2020-11-25T01:29:41Z)
• A Compressive Sensing Approach for Federated Learning over Massive MIMO Communication Systems [82.2513703281725]
  Federated learning is a privacy-preserving approach to train a global model at a central server by collaborating with wireless devices. We present a compressive sensing approach for federated learning over massive multiple-input multiple-output communication systems.
  arXiv  Detail & Related papers  (2020-03-18T05:56:27Z)
• Federated Learning with Cooperating Devices: A Consensus Approach for Massive IoT Networks [8.456633924613456]
  Federated learning (FL) is emerging as a new paradigm to train machine learning models in distributed systems. The paper proposes a fully distributed (or server-less) learning approach: the proposed FL algorithms leverage the cooperation of devices that perform data operations inside the network. The approach lays the groundwork for integration of FL within 5G and beyond networks characterized by decentralized connectivity and computing.
  arXiv  Detail & Related papers  (2019-12-27T15:16:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.