Learning to Sail Dynamic Networks: The MARLIN Reinforcement Learning
Framework for Congestion Control in Tactical Environments
- URL: http://arxiv.org/abs/2306.15591v1
- Date: Tue, 27 Jun 2023 16:15:15 GMT
- Title: Learning to Sail Dynamic Networks: The MARLIN Reinforcement Learning
Framework for Congestion Control in Tactical Environments
- Authors: Raffaele Galliera, Mattia Zaccarini, Alessandro Morelli, Roberto
Fronteddu, Filippo Poltronieri, Niranjan Suri, Mauro Tortonesi
- Abstract summary: This paper proposes an RL framework that leverages an accurate and parallelizable emulation environment to reenact the conditions of a tactical network.
We evaluate our RL learning framework by training a MARLIN agent in conditions replicating a bottleneck link transition between a Satellite Communication (SATCOM) and an UHF Wide Band (UHF) radio link.
- Score: 53.08686495706487
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Conventional Congestion Control (CC) algorithms,such as TCP Cubic, struggle
in tactical environments as they misinterpret packet loss and fluctuating
network performance as congestion symptoms. Recent efforts, including our own
MARLIN, have explored the use of Reinforcement Learning (RL) for CC, but they
often fall short of generalization, particularly in competitive, unstable, and
unforeseen scenarios. To address these challenges, this paper proposes an RL
framework that leverages an accurate and parallelizable emulation environment
to reenact the conditions of a tactical network. We also introduce refined RL
formulation and performance evaluation methods tailored for agents operating in
such intricate scenarios. We evaluate our RL learning framework by training a
MARLIN agent in conditions replicating a bottleneck link transition between a
Satellite Communication (SATCOM) and an UHF Wide Band (UHF) radio link.
Finally, we compared its performance in file transfer tasks against
Transmission Control Protocol (TCP) Cubic and the default strategy implemented
in the Mockets tactical communication middleware. The results demonstrate that
the MARLIN RL agent outperforms both TCP and Mockets under different
perspectives and highlight the effectiveness of specialized RL solutions in
optimizing CC for tactical network environments.
Related papers
- Enhancing Spectrum Efficiency in 6G Satellite Networks: A GAIL-Powered Policy Learning via Asynchronous Federated Inverse Reinforcement Learning [67.95280175998792]
A novel adversarial imitation learning (GAIL)-powered policy learning approach is proposed for optimizing beamforming, spectrum allocation, and remote user equipment (RUE) association ins.
We employ inverse RL (IRL) to automatically learn reward functions without manual tuning.
We show that the proposed MA-AL method outperforms traditional RL approaches, achieving a $14.6%$ improvement in convergence and reward value.
arXiv Detail & Related papers (2024-09-27T13:05:02Z) - iLLM-TSC: Integration reinforcement learning and large language model for traffic signal control policy improvement [5.078593258867346]
We introduce a novel integration framework that combines a large language model (LLM) with reinforcement learning (RL)
Our approach reduces the average waiting time by $17.5%$ in degraded communication conditions as compared to traditional RL methods.
arXiv Detail & Related papers (2024-07-08T15:22:49Z) - MARLIN: Soft Actor-Critic based Reinforcement Learning for Congestion
Control in Real Networks [63.24965775030673]
We propose a novel Reinforcement Learning (RL) approach to design generic Congestion Control (CC) algorithms.
Our solution, MARLIN, uses the Soft Actor-Critic algorithm to maximize both entropy and return.
We trained MARLIN on a real network with varying background traffic patterns to overcome the sim-to-real mismatch.
arXiv Detail & Related papers (2023-02-02T18:27:20Z) - Mastering the Unsupervised Reinforcement Learning Benchmark from Pixels [112.63440666617494]
Reinforcement learning algorithms can succeed but require large amounts of interactions between the agent and the environment.
We propose a new method to solve it, using unsupervised model-based RL, for pre-training the agent.
We show robust performance on the Real-Word RL benchmark, hinting at resiliency to environment perturbations during adaptation.
arXiv Detail & Related papers (2022-09-24T14:22:29Z) - Curriculum Based Reinforcement Learning of Grid Topology Controllers to
Prevent Thermal Cascading [0.19116784879310028]
This paper describes how domain knowledge of power system operators can be integrated into reinforcement learning frameworks.
A curriculum-based approach with reward tuning is incorporated into the training procedure by modifying the environment.
A parallel training approach on multiple scenarios is employed to avoid biasing the agent to a few scenarios and make it robust to the natural variability in grid operations.
arXiv Detail & Related papers (2021-12-18T20:32:05Z) - Vehicular Cooperative Perception Through Action Branching and Federated
Reinforcement Learning [101.64598586454571]
A novel framework is proposed to allow reinforcement learning-based vehicular association, resource block (RB) allocation, and content selection of cooperative perception messages (CPMs)
A federated RL approach is introduced in order to speed up the training process across vehicles.
Results show that federated RL improves the training process, where better policies can be achieved within the same amount of time compared to the non-federated approach.
arXiv Detail & Related papers (2020-12-07T02:09:15Z) - Pareto Deterministic Policy Gradients and Its Application in 5G Massive
MIMO Networks [32.099949375036495]
We consider jointly optimizing cell load balance and network throughput via a reinforcement learning (RL) approach.
Our rationale behind using RL is to circumvent the challenges of analytically modeling user mobility and network dynamics.
To accomplish this joint optimization, we integrate vector rewards into the RL value network and conduct RL action via a separate policy network.
arXiv Detail & Related papers (2020-12-02T15:35:35Z) - Robust Deep Reinforcement Learning through Adversarial Loss [74.20501663956604]
Recent studies have shown that deep reinforcement learning agents are vulnerable to small adversarial perturbations on the agent's inputs.
We propose RADIAL-RL, a principled framework to train reinforcement learning agents with improved robustness against adversarial attacks.
arXiv Detail & Related papers (2020-08-05T07:49:42Z) - Multi-agent Reinforcement Learning for Networked System Control [6.89105475513757]
This paper considers multi-agent reinforcement learning (MARL) in networked system control.
We propose a new different communication protocol, called NeurComm, to reduce information loss and non-stationarity in NMARL.
NeurComm outperforms existing communication protocols in both learning efficiency and control performance.
arXiv Detail & Related papers (2020-04-03T02:21:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.