SMAClite: A Lightweight Environment for Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2305.05566v1
• Date: Tue, 9 May 2023 15:55:19 GMT
• Title: SMAClite: A Lightweight Environment for Multi-Agent Reinforcement Learning
• Authors: Adam Michalski, Filippos Christianos, Stefano V. Albrecht
• Abstract summary: The Starcraft Multi-Agent Challenge (SMAC) has been widely used in MARL research, but is built on top of a heavy, closed-source computer game, StarCraft II. We introduce SMAClite -- a challenge based on SMAC that is both decoupled from Starcraft II and open-source, along with a framework which makes it possible to create new content for SMAClite without any special knowledge. We conduct experiments to show that SMAClite is equivalent to SMAC, by training MARL algorithms on SMAClite and reproducing SMAC results.
• Score: 11.292086312664383
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: There is a lack of standard benchmarks for Multi-Agent Reinforcement Learning (MARL) algorithms. The Starcraft Multi-Agent Challenge (SMAC) has been widely used in MARL research, but is built on top of a heavy, closed-source computer game, StarCraft II. Thus, SMAC is computationally expensive and requires knowledge and the use of proprietary tools specific to the game for any meaningful alteration or contribution to the environment. We introduce SMAClite -- a challenge based on SMAC that is both decoupled from Starcraft II and open-source, along with a framework which makes it possible to create new content for SMAClite without any special knowledge. We conduct experiments to show that SMAClite is equivalent to SMAC, by training MARL algorithms on SMAClite and reproducing SMAC results. We then show that SMAClite outperforms SMAC in both runtime speed and memory.

Related papers

• A New Approach to Solving SMAC Task: Generating Decision Tree Code from Large Language Models [8.457552813123597]
  StarCraft Multi-Agent Challenge (SMAC) is one of the most commonly used experimental environments in multi-agent reinforcement learning (MARL) Traditional MARL algorithms often require interacting with the environment for up to 1 million steps to train a model. In this paper, we propose a novel approach to solving SMAC tasks called LLM-SMAC.
  arXiv  Detail & Related papers  (2024-10-21T13:58:38Z)
• JaxMARL: Multi-Agent RL Environments and Algorithms in JAX [105.343918678781]
  We present JaxMARL, the first open-source, Python-based library that combines GPU-enabled efficiency with support for a large number of commonly used MARL environments. Our experiments show that, in terms of wall clock time, our JAX-based training pipeline is around 14 times faster than existing approaches. We also introduce and benchmark SMAX, a JAX-based approximate reimplementation of the popular StarCraft Multi-Agent Challenge.
  arXiv  Detail & Related papers  (2023-11-16T18:58:43Z)
• Towards Semantic Communication Protocols for 6G: From Protocol Learning to Language-Oriented Approaches [60.6632432485476]
  6G systems are expected to address a wide range of non-stationary tasks. This poses challenges to traditional medium access control (MAC) protocols that are static and predefined. Data-driven MAC protocols have recently emerged, offering ability to tailor their signaling messages for specific tasks. This article presents a novel categorization of these data-driven MAC protocols into three levels: Level 1 MAC. task-oriented neural protocols constructed using multi-agent deep reinforcement learning (MADRL); Level 2 MAC. neural network-oriented symbolic protocols developed by converting Level 1 MAC outputs into explicit symbols; and Level 3 MAC. language-oriented semantic protocols harnessing
  arXiv  Detail & Related papers  (2023-10-14T06:28:50Z)
• L2MAC: Large Language Model Automatic Computer for Extensive Code Generation [52.81694565226513]
  Transformer-based large language models (LLMs) are constrained by the fixed context window of the underlying transformer architecture. This paper presents L2MAC, the first practical LLM-based general-purpose stored-program automatic computer (von Neumann architecture) framework, for long and consistent output generation.
  arXiv  Detail & Related papers  (2023-10-02T16:55:19Z)
• IMAC-Sim: A Circuit-level Simulator For In-Memory Analog Computing Architectures [0.0]
  IMAC-Sim is a circuit-level simulator for the design space exploration of IMAC architectures. IMAC-Sim is a Python-based simulation framework, which creates the SPICE netlist of the IMAC circuit.
  arXiv  Detail & Related papers  (2023-04-18T19:22:34Z)
• SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learning [45.98103968842858]
  The StarCraft Multi-Agent Challenge (SMAC) is a popular testbed for centralised training with decentralised execution. We show that SMAC lacks the partial observability to require complex *closed-loop* policies. We introduce SMACv2, a new version of the benchmark where scenarios are procedurally generated and require agents to generalise to previously unseen settings.
  arXiv  Detail & Related papers  (2022-12-14T20:15:19Z)
• Extending Compositional Attention Networks for Social Reasoning in Videos [84.12658971655253]
  We propose a novel deep architecture for the task of reasoning about social interactions in videos. We leverage the multi-step reasoning capabilities of Compositional Attention Networks (MAC), and propose a multimodal extension (MAC-X)
  arXiv  Detail & Related papers  (2022-10-03T19:03:01Z)
• Transformer-based Value Function Decomposition for Cooperative Multi-agent Reinforcement Learning in StarCraft [1.160208922584163]
  The StarCraft II Multi-Agent Challenge (SMAC) was created to be a benchmark problem for cooperative multi-agent reinforcement learning (MARL) This paper introduces a new architecture TransMix, a transformer-based joint action-value mixing network.
  arXiv  Detail & Related papers  (2022-08-15T16:13:16Z)
• Divergence-Regularized Multi-Agent Actor-Critic [17.995905582226467]
  We propose a novel off-policy cooperative MARL framework, divergence-regularized multi-agent actor-critic (DMAC) DMAC is a flexible framework and can be combined with many existing MARL algorithms. We empirically show that DMAC substantially improves the performance of existing MARL algorithms.
  arXiv  Detail & Related papers  (2021-10-01T10:27:42Z)
• MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius [133.47492985863136]
  Adversarial training is one of the most popular ways to learn robust models but is usually attack-dependent and time costly. We propose the MACER algorithm, which learns robust models without using adversarial training but performs better than all existing provable l2-defenses. For all tasks, MACER spends less training time than state-of-the-art adversarial training algorithms, and the learned models achieve larger average certified radius.
  arXiv  Detail & Related papers  (2020-01-08T05:08:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.