Distributed Influence-Augmented Local Simulators for Parallel MARL in
Large Networked Systems
- URL: http://arxiv.org/abs/2207.00288v2
- Date: Fri, 1 Mar 2024 08:36:33 GMT
- Title: Distributed Influence-Augmented Local Simulators for Parallel MARL in
Large Networked Systems
- Authors: Miguel Suau, Jinke He, Mustafa Mert \c{C}elikok, Matthijs T. J. Spaan,
Frans A. Oliehoek
- Abstract summary: We show how to decompose large networked systems of many agents into multiple local components.
To monitor the influence that the different local components exert on one another, each of these simulators is equipped with a learned model.
- Score: 21.82191916743004
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Due to its high sample complexity, simulation is, as of today, critical for
the successful application of reinforcement learning. Many real-world problems,
however, exhibit overly complex dynamics, which makes their full-scale
simulation computationally slow. In this paper, we show how to decompose large
networked systems of many agents into multiple local components such that we
can build separate simulators that run independently and in parallel. To
monitor the influence that the different local components exert on one another,
each of these simulators is equipped with a learned model that is periodically
trained on real trajectories. Our empirical results reveal that distributing
the simulation among different processes not only makes it possible to train
large multi-agent systems in just a few hours but also helps mitigate the
negative effects of simultaneous learning.
Related papers
- LASIL: Learner-Aware Supervised Imitation Learning For Long-term Microscopic Traffic Simulation [30.500368103677097]
Microscopic traffic simulation plays a crucial role in transportation engineering by providing insights into individual vehicle behavior and overall traffic flow.
Traditional simulators relying on models often fail to deliver accurate simulations due to the complexity of real-world traffic environments.
In this paper, we propose a novel approach called learner-aware supervised imitation learning.
arXiv Detail & Related papers (2024-03-26T11:13:35Z) - Towards Complex Dynamic Physics System Simulation with Graph Neural ODEs [75.7104463046767]
This paper proposes a novel learning based simulation model that characterizes the varying spatial and temporal dependencies in particle systems.
We empirically evaluate GNSTODE's simulation performance on two real-world particle systems, Gravity and Coulomb.
arXiv Detail & Related papers (2023-05-21T03:51:03Z) - Hindsight States: Blending Sim and Real Task Elements for Efficient
Reinforcement Learning [61.3506230781327]
In robotics, one approach to generate training data builds on simulations based on dynamics models derived from first principles.
Here, we leverage the imbalance in complexity of the dynamics to learn more sample-efficiently.
We validate our method on several challenging simulated tasks and demonstrate that it improves learning both alone and when combined with an existing hindsight algorithm.
arXiv Detail & Related papers (2023-03-03T21:55:04Z) - Simulation-Based Parallel Training [55.41644538483948]
We present our ongoing work to design a training framework that alleviates those bottlenecks.
It generates data in parallel with the training process.
We present a strategy to mitigate this bias with a memory buffer.
arXiv Detail & Related papers (2022-11-08T09:31:25Z) - Sim and Real: Better Together [47.14469055555684]
We demonstrate how to learn simultaneously from both simulation and interaction with the real environment.
We propose an algorithm for balancing the large number of samples from the high throughput but less accurate simulation.
We analyze such multi-environment interaction theoretically, and provide convergence properties, through a novel theoretical replay buffer analysis.
arXiv Detail & Related papers (2021-10-01T14:30:03Z) - TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors [74.67698916175614]
We propose TrafficSim, a multi-agent behavior model for realistic traffic simulation.
In particular, we leverage an implicit latent variable model to parameterize a joint actor policy.
We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines.
arXiv Detail & Related papers (2021-01-17T00:29:30Z) - From Simulation to Real World Maneuver Execution using Deep
Reinforcement Learning [69.23334811890919]
Deep Reinforcement Learning has proved to be able to solve many control tasks in different fields, but the behavior of these systems is not always as expected when deployed in real-world scenarios.
This is mainly due to the lack of domain adaptation between simulated and real-world data together with the absence of distinction between train and test datasets.
We present a system based on multiple environments in which agents are trained simultaneously, evaluating the behavior of the model in different scenarios.
arXiv Detail & Related papers (2020-05-13T14:22:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.