Hierarchical Multi-Agent Multi-Armed Bandit for Resource Allocation in Multi-LEO Satellite Constellation Networks

• URL: http://arxiv.org/abs/2303.14351v1
• Date: Sat, 25 Mar 2023 04:22:07 GMT
• Title: Hierarchical Multi-Agent Multi-Armed Bandit for Resource Allocation in Multi-LEO Satellite Constellation Networks
• Authors: Li-Hsiang Shen, Yun Ho, Kai-Ten Feng, Lie-Liang Yang, Sau-Hsuan Wu, Jen-Ming Wu
• Abstract summary: Low Earth orbit (LEO) satellite constellation is capable of providing global coverage area with high-rate services. We propose hierarchical multi-agent multi-armed bandit resource allocation for LEO constellation (mmRAL) by appropriately assigning available radio resources.
• Score: 14.964082610286857
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Low Earth orbit (LEO) satellite constellation is capable of providing global coverage area with high-rate services in the next sixth-generation (6G) non-terrestrial network (NTN). Due to limited onboard resources of operating power, beams, and channels, resilient and efficient resource management has become compellingly imperative under complex interference cases. However, different from conventional terrestrial base stations, LEO is deployed at considerable height and under high mobility, inducing substantially long delay and interference during transmission. As a result, acquiring the accurate channel state information between LEOs and ground users is challenging. Therefore, we construct a framework with a two-way transmission under unknown channel information and no data collected at long-delay ground gateway. In this paper, we propose hierarchical multi-agent multi-armed bandit resource allocation for LEO constellation (mmRAL) by appropriately assigning available radio resources. LEOs are considered as collaborative multiple macro-agents attempting unknown trials of various actions of micro-agents of respective resources, asymptotically achieving suitable allocation with only throughput information. In simulations, we evaluate mmRAL in various cases of LEO deployment, serving numbers of users and LEOs, hardware cost and outage probability. Benefited by efficient and resilient allocation, the proposed mmRAL system is capable of operating in homogeneous or heterogeneous orbital planes or constellations, achieving the highest throughput performance compared to the existing benchmarks in open literature.

Related papers

• Tera-SpaceCom: GNN-based Deep Reinforcement Learning for Joint Resource Allocation and Task Offloading in TeraHertz Band Space Networks [11.881917133887102]
  Tera-SpaceCom is envisioned as a promising technology to enable various space science and communication applications. A graph neural network (GNN)-deep reinforcement learning (DRL)-based joint resource allocation and task offloading algorithm is proposed.
  arXiv  Detail & Related papers  (2024-09-12T10:26:17Z)
• WDMoE: Wireless Distributed Large Language Models with Mixture of Experts [65.57581050707738]
  We propose a wireless distributed Large Language Models (LLMs) paradigm based on Mixture of Experts (MoE) We decompose the MoE layer in LLMs by deploying the gating network and the preceding neural network layer at base station (BS) and mobile devices. We design an expert selection policy by taking into account both the performance of the model and the end-to-end latency.
  arXiv  Detail & Related papers  (2024-05-06T02:55:50Z)
• Collaborative Ground-Space Communications via Evolutionary Multi-objective Deep Reinforcement Learning [113.48727062141764]
  We propose a distributed collaborative beamforming (DCB)-based uplink communication paradigm for enabling ground-space direct communications. DCB treats the terminals that are unable to establish efficient direct connections with the low Earth orbit (LEO) satellites as distributed antennas. We propose an evolutionary multi-objective deep reinforcement learning algorithm to obtain the desirable policies.
  arXiv  Detail & Related papers  (2024-04-11T03:13:02Z)
• Privacy-Aware Spectrum Pricing and Power Control Optimization for LEO Satellite Internet-of-Things [14.706902417174039]
  We propose a hybrid spectrum pricing and power control framework for LEO IoT. We first design a local deep reinforcement learning algorithm for LEO satellite systems to learn a revenue-maximizing pricing and power control scheme. We also propose a reputation-based blockchain which is used in the global model aggregation phase of FL.
  arXiv  Detail & Related papers  (2024-04-01T09:15:48Z)
• A Distributed Neural Linear Thompson Sampling Framework to Achieve URLLC in Industrial IoT [16.167107624956294]
  Industrial Internet of Things (IIoT) networks will provide Ultra-Reliable Low-Latency Communication (URLLC) to support critical processes. Standard protocols for allocating wireless resources may not optimize the latency-reliability trade-off, especially for uplink communication.
  arXiv  Detail & Related papers  (2023-11-21T12:22:04Z)
• Cooperative Multi-Type Multi-Agent Deep Reinforcement Learning for Resource Management in Space-Air-Ground Integrated Networks [10.351443157142295]
  We develop a comprehensive SAGIN system that encompasses five distinct communication links. We propose an efficient cooperative multi-type multi-agent deep reinforcement learning (CMT-MARL) method to address the resource management issue.
  arXiv  Detail & Related papers  (2023-08-08T02:15:06Z)
• Edge Selection and Clustering for Federated Learning in Optical Inter-LEO Satellite Constellation [12.489681058742358]
  We have proposed a collaborative federated learning for low Earth orbit (FELLO) We allocate the entire process on LEOs with low payload inter-satellite transmissions, whilst the low-delay terrestrial gateway server (GS) only takes care for initial signal controlling. The proposed FELLO outperforms the conventional centralized and distributed architectures with higher classification accuracy as well as comparably lower latency of joint communication and computing.
  arXiv  Detail & Related papers  (2023-03-25T04:24:16Z)
• Artificial Intelligence Empowered Multiple Access for Ultra Reliable and Low Latency THz Wireless Networks [76.89730672544216]
  Terahertz (THz) wireless networks are expected to catalyze the beyond fifth generation (B5G) era. To satisfy the ultra-reliability and low-latency demands of several B5G applications, novel mobility management approaches are required. This article presents a holistic MAC layer approach that enables intelligent user association and resource allocation, as well as flexible and adaptive mobility management.
  arXiv  Detail & Related papers  (2022-08-17T03:00:24Z)
• Learning Emergent Random Access Protocol for LEO Satellite Networks [51.575090080749554]
  We propose a novel grant-free random access solution for LEO SAT networks, dubbed emergent random access channel protocol (eRACH) eRACH is a model-free approach that emerges through interaction with the non-stationary network environment. Compared to RACH, we show from various simulations that our proposed eRACH yields 54.6% higher average network throughput.
  arXiv  Detail & Related papers  (2021-12-03T07:44:45Z)
• Locality Matters: A Scalable Value Decomposition Approach for Cooperative Multi-Agent Reinforcement Learning [52.7873574425376]
  Cooperative multi-agent reinforcement learning (MARL) faces significant scalability issues due to state and action spaces that are exponentially large in the number of agents. We propose a novel, value-based multi-agent algorithm called LOMAQ, which incorporates local rewards in the Training Decentralized Execution paradigm.
  arXiv  Detail & Related papers  (2021-09-22T10:08:15Z)
• Integrating LEO Satellite and UAV Relaying via Reinforcement Learning for Non-Terrestrial Networks [51.05735925326235]
  A mega-constellation of low-earth orbit (LEO) satellites has the potential to enable long-range communication with low latency. We study the problem of forwarding packets between two faraway ground terminals, through an LEO satellite selected from an orbiting constellation. To maximize the end-to-end data rate, the satellite association and HAP location should be optimized. We tackle this problem using deep reinforcement learning (DRL) with a novel action dimension reduction technique.
  arXiv  Detail & Related papers  (2020-05-26T05:39:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.