Large-Scale Multi-Agent Deep FBSDEs

• URL: http://arxiv.org/abs/2011.10890v3
• Date: Fri, 21 May 2021 04:46:01 GMT
• Title: Large-Scale Multi-Agent Deep FBSDEs
• Authors: Tianrong Chen, Ziyi Wang, Ioannis Exarchos, Evangelos A. Theodorou
• Abstract summary: We present a framework for finding Markovian Nash Equilibria in multi-agent games using fictitious play. We showcase superior performance of our framework over the state-of-the-art deep fictitious play algorithm. We also demonstrate the applicability of our framework in robotics on a belief space autonomous racing problem.
• Score: 28.525065041507982
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper we present a scalable deep learning framework for finding Markovian Nash Equilibria in multi-agent stochastic games using fictitious play. The motivation is inspired by theoretical analysis of Forward Backward Stochastic Differential Equations (FBSDE) and their implementation in a deep learning setting, which is the source of our algorithm's sample efficiency improvement. By taking advantage of the permutation-invariant property of agents in symmetric games, the scalability and performance is further enhanced significantly. We showcase superior performance of our framework over the state-of-the-art deep fictitious play algorithm on an inter-bank lending/borrowing problem in terms of multiple metrics. More importantly, our approach scales up to 3000 agents in simulation, a scale which, to the best of our knowledge, represents a new state-of-the-art. We also demonstrate the applicability of our framework in robotics on a belief space autonomous racing problem.

Related papers

• BAIT: Benchmarking (Embedding) Architectures for Interactive Theorem-Proving [13.374504717801061]
  We present BAIT, a framework for fair and streamlined comparison of learning approaches in Interactive Theorem Proving. We demonstrate BAIT's capabilities with an in-depth comparison, across several ITP benchmarks. BAIT also allows us to assess the end-to-end proving performance of systems built on interactive environments.
  arXiv  Detail & Related papers  (2024-03-06T01:56:17Z)
• Scalable Semantic Non-Markovian Simulation Proxy for Reinforcement Learning [0.125828876338076]
  We propose a semantic proxy for simulation based on a temporal extension to annotated logic. We show up to three orders of magnitude speed-up while preserving the quality of policy learned.
  arXiv  Detail & Related papers  (2023-10-10T17:59:26Z)
• Guaranteed Conservation of Momentum for Learning Particle-based Fluid Dynamics [96.9177297872723]
  We present a novel method for guaranteeing linear momentum in learned physics simulations. We enforce conservation of momentum with a hard constraint, which we realize via antisymmetrical continuous convolutional layers. In combination, the proposed method allows us to increase the physical accuracy of the learned simulator substantially.
  arXiv  Detail & Related papers  (2022-10-12T09:12:59Z)
• Multi-Agent Reinforcement Learning for Long-Term Network Resource Allocation through Auction: a V2X Application [7.326507804995567]
  We formulate offloading of computational tasks from a dynamic group of mobile agents (e.g., cars) as decentralized decision making among autonomous agents. We design an interaction mechanism that incentivizes such agents to align private and system goals by balancing between competition and cooperation. We propose a novel multi-agent online learning algorithm that learns with partial, delayed and noisy state information.
  arXiv  Detail & Related papers  (2022-07-29T10:29:06Z)
• Slimmable Domain Adaptation [112.19652651687402]
  We introduce a simple framework, Slimmable Domain Adaptation, to improve cross-domain generalization with a weight-sharing model bank. Our framework surpasses other competing approaches by a very large margin on multiple benchmarks.
  arXiv  Detail & Related papers  (2022-06-14T06:28:04Z)
• Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computation [93.52573037053449]
  H-MARL (Hallucinated Multi-Agent Reinforcement Learning) learns successful equilibrium policies after a few interactions with the environment. We demonstrate our approach experimentally on an autonomous driving simulation benchmark.
  arXiv  Detail & Related papers  (2022-03-14T17:24:03Z)
• Efficient Differentiable Simulation of Articulated Bodies [89.64118042429287]
  We present a method for efficient differentiable simulation of articulated bodies. This enables integration of articulated body dynamics into deep learning frameworks. We show that reinforcement learning with articulated systems can be accelerated using gradients provided by our method.
  arXiv  Detail & Related papers  (2021-09-16T04:48:13Z)
• Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learning [89.31889875864599]
  We propose an efficient model-based reinforcement learning algorithm for learning in multi-agent systems. Our main theoretical contributions are the first general regret bounds for model-based reinforcement learning for MFC. We provide a practical parametrization of the core optimization problem.
  arXiv  Detail & Related papers  (2021-07-08T18:01:02Z)
• Softmax with Regularization: Better Value Estimation in Multi-Agent Reinforcement Learning [72.28520951105207]
  Overestimation in $Q$-learning is an important problem that has been extensively studied in single-agent reinforcement learning. We propose a novel regularization-based update scheme that penalizes large joint action-values deviating from a baseline. We show that our method provides a consistent performance improvement on a set of challenging StarCraft II micromanagement tasks.
  arXiv  Detail & Related papers  (2021-03-22T14:18:39Z)
• Non-cooperative Multi-agent Systems with Exploring Agents [10.736626320566707]
  We develop a prescriptive model of multi-agent behavior using Markov games. We focus on models in which the agents play "exploration but near optimum strategies"
  arXiv  Detail & Related papers  (2020-05-25T19:34:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.