Off-the-Grid MARL: Datasets with Baselines for Offline Multi-Agent
Reinforcement Learning
- URL: http://arxiv.org/abs/2302.00521v2
- Date: Fri, 22 Sep 2023 19:25:31 GMT
- Title: Off-the-Grid MARL: Datasets with Baselines for Offline Multi-Agent
Reinforcement Learning
- Authors: Claude Formanek, Asad Jeewa, Jonathan Shock, Arnu Pretorius
- Abstract summary: offline multi-agent reinforcement learning (MARL) provides a promising paradigm for building effective decentralised controllers from such datasets.
MARL is still in its infancy and therefore lacks standardised benchmark datasets and baselines.
OG-MARL is a growing repository of high-quality datasets with baselines for cooperative offline MARL research.
- Score: 4.159549932951023
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Being able to harness the power of large datasets for developing cooperative
multi-agent controllers promises to unlock enormous value for real-world
applications. Many important industrial systems are multi-agent in nature and
are difficult to model using bespoke simulators. However, in industry,
distributed processes can often be recorded during operation, and large
quantities of demonstrative data stored. Offline multi-agent reinforcement
learning (MARL) provides a promising paradigm for building effective
decentralised controllers from such datasets. However, offline MARL is still in
its infancy and therefore lacks standardised benchmark datasets and baselines
typically found in more mature subfields of reinforcement learning (RL). These
deficiencies make it difficult for the community to sensibly measure progress.
In this work, we aim to fill this gap by releasing off-the-grid MARL (OG-MARL):
a growing repository of high-quality datasets with baselines for cooperative
offline MARL research. Our datasets provide settings that are characteristic of
real-world systems, including complex environment dynamics, heterogeneous
agents, non-stationarity, many agents, partial observability, suboptimality,
sparse rewards and demonstrated coordination. For each setting, we provide a
range of different dataset types (e.g. Good, Medium, Poor, and Replay) and
profile the composition of experiences for each dataset. We hope that OG-MARL
will serve the community as a reliable source of datasets and help drive
progress, while also providing an accessible entry point for researchers new to
the field.
Related papers
- The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective [53.48484062444108]
We find that the development of models and data is not two separate paths but rather interconnected.
On the one hand, vaster and higher-quality data contribute to better performance of MLLMs, on the other hand, MLLMs can facilitate the development of data.
To promote the data-model co-development for MLLM community, we systematically review existing works related to MLLMs from the data-model co-development perspective.
arXiv Detail & Related papers (2024-07-11T15:08:11Z) - DCA-Bench: A Benchmark for Dataset Curation Agents [9.60250892491588]
We propose a dataset curation agent benchmark, DCA-Bench, to measure large language models' capability of detecting hidden dataset quality issues.
Specifically, we collect diverse real-world dataset quality issues from eight open dataset platforms as a testbed.
The proposed benchmark can also serve as a testbed for measuring the capability of LLMs in problem discovery rather than just problem-solving.
arXiv Detail & Related papers (2024-06-11T14:02:23Z) - MABL: Bi-Level Latent-Variable World Model for Sample-Efficient
Multi-Agent Reinforcement Learning [43.30657890400801]
We propose a novel model-based MARL algorithm, MABL, that learns a bi-level latent-variable world model from high-dimensional inputs.
For each agent, MABL learns a global latent state at the upper level, which is used to inform the learning of an agent latent state at the lower level.
MaBL surpasses SOTA multi-agent latent-variable world models in both sample efficiency and overall performance.
arXiv Detail & Related papers (2023-04-12T17:46:23Z) - Learning From Good Trajectories in Offline Multi-Agent Reinforcement
Learning [98.07495732562654]
offline multi-agent reinforcement learning (MARL) aims to learn effective multi-agent policies from pre-collected datasets.
One agent learned by offline MARL often inherits this random policy, jeopardizing the performance of the entire team.
We propose a novel framework called Shared Individual Trajectories (SIT) to address this problem.
arXiv Detail & Related papers (2022-11-28T18:11:26Z) - TRoVE: Transforming Road Scene Datasets into Photorealistic Virtual
Environments [84.6017003787244]
This work proposes a synthetic data generation pipeline to address the difficulties and domain-gaps present in simulated datasets.
We show that using annotations and visual cues from existing datasets, we can facilitate automated multi-modal data generation.
arXiv Detail & Related papers (2022-08-16T20:46:08Z) - DataPerf: Benchmarks for Data-Centric AI Development [81.03754002516862]
DataPerf is a community-led benchmark suite for evaluating ML datasets and data-centric algorithms.
We provide an open, online platform with multiple rounds of challenges to support this iterative development.
The benchmarks, online evaluation platform, and baseline implementations are open source.
arXiv Detail & Related papers (2022-07-20T17:47:54Z) - Collaborative Visual Navigation [69.20264563368762]
We propose a large-scale 3D dataset, CollaVN, for multi-agent visual navigation (MAVN)
Diverse MAVN variants are explored to make our problem more general.
A memory-augmented communication framework is proposed. Each agent is equipped with a private, external memory to persistently store communication information.
arXiv Detail & Related papers (2021-07-02T15:48:16Z) - More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing
Imagery Classification [43.35966675372692]
We show how to train deep networks and build the network architecture.
In particular, we show different fusion strategies as well as how to train deep networks and build the network architecture.
Our framework is not only limited to pixel-wise classification tasks but also applicable to spatial information modeling with convolutional neural networks (CNNs)
arXiv Detail & Related papers (2020-08-12T17:45:25Z) - D4RL: Datasets for Deep Data-Driven Reinforcement Learning [119.49182500071288]
We introduce benchmarks specifically designed for the offline setting, guided by key properties of datasets relevant to real-world applications of offline RL.
By moving beyond simple benchmark tasks and data collected by partially-trained RL agents, we reveal important and unappreciated deficiencies of existing algorithms.
arXiv Detail & Related papers (2020-04-15T17:18:19Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.