Extended Markov Games to Learn Multiple Tasks in Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2002.06000v1
• Date: Fri, 14 Feb 2020 12:37:41 GMT
• Title: Extended Markov Games to Learn Multiple Tasks in Multi-Agent Reinforcement Learning
• Authors: Borja G. Le\'on and Francesco Belardinelli
• Abstract summary: We formally define Extended Markov Games as a general mathematical model that allows multiple RL agents to concurrently learn various non-Markovian specifications. Specifically, we use our model to train two different logic-based multi-agent RL algorithms to solve diverse settings of non-Markovian co-safe specifications.
• Score: 7.332887338089177
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The combination of Formal Methods with Reinforcement Learning (RL) has recently attracted interest as a way for single-agent RL to learn multiple-task specifications. In this paper we extend this convergence to multi-agent settings and formally define Extended Markov Games as a general mathematical model that allows multiple RL agents to concurrently learn various non-Markovian specifications. To introduce this new model we provide formal definitions and proofs as well as empirical tests of RL algorithms running on this framework. Specifically, we use our model to train two different logic-based multi-agent RL algorithms to solve diverse settings of non-Markovian co-safe LTL specifications.

Related papers

• MOMAland: A Set of Benchmarks for Multi-Objective Multi-Agent Reinforcement Learning [7.822825134714791]
  Multi-objective multi-agent reinforcement learning (MOMARL) addresses problems with multiple agents each needing to consider multiple objectives in their learning process. MOAland is the first collection of standardised environments for multi-objective multi-agent reinforcement learning.
  arXiv  Detail & Related papers  (2024-07-23T09:05:06Z)
• ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL [80.10358123795946]
  We develop a framework for building multi-turn RL algorithms for fine-tuning large language models. Our framework adopts a hierarchical RL approach and runs two RL algorithms in parallel. Empirically, we find that ArCHer significantly improves efficiency and performance on agent tasks.
  arXiv  Detail & Related papers  (2024-02-29T18:45:56Z)
• Model-Based RL for Mean-Field Games is not Statistically Harder than Single-Agent RL [57.745700271150454]
  We study the sample complexity of reinforcement learning in Mean-Field Games (MFGs) with model-based function approximation. We introduce the Partial Model-Based Eluder Dimension (P-MBED), a more effective notion to characterize the model class complexity.
  arXiv  Detail & Related papers  (2024-02-08T14:54:47Z)
• MADiff: Offline Multi-agent Learning with Diffusion Models [79.18130544233794]
  Diffusion model (DM) recently achieved huge success in various scenarios including offline reinforcement learning. We propose MADiff, a novel generative multi-agent learning framework to tackle this problem. Our experiments show the superior performance of MADiff compared to baseline algorithms in a wide range of multi-agent learning tasks.
  arXiv  Detail & Related papers  (2023-05-27T02:14:09Z)
• Centralized control for multi-agent RL in a complex Real-Time-Strategy game [0.0]
  Multi-agent Reinforcement learning (MARL) studies the behaviour of multiple learning agents that coexist in a shared environment. MARL is more challenging than single-agent RL because it involves more complex learning dynamics. This project provides the end-to-end experience of applying RL in the Lux AI v2 Kaggle competition.
  arXiv  Detail & Related papers  (2023-04-25T17:19:05Z)
• Multi-Agent Reinforcement Learning for Microprocessor Design Space Exploration [71.95914457415624]
  Microprocessor architects are increasingly resorting to domain-specific customization in the quest for high-performance and energy-efficiency. We propose an alternative formulation that leverages Multi-Agent RL (MARL) to tackle this problem. Our evaluation shows that the MARL formulation consistently outperforms single-agent RL baselines.
  arXiv  Detail & Related papers  (2022-11-29T17:10:24Z)
• The Power of Exploiter: Provable Multi-Agent RL in Large State Spaces [36.097537237660234]
  We propose an algorithm that can provably find the Nash equilibrium policy using a number of samples. A key component of our new algorithm is the exploiter, which facilitates the learning of the main player by deliberately exploiting her weakness. Our theoretical framework is generic, which applies to a wide range of models including but not limited to MGs, MGs with linear or kernel function approximation, and MGs with rich observations.
  arXiv  Detail & Related papers  (2021-06-07T05:39:09Z)
• Non-cooperative Multi-agent Systems with Exploring Agents [10.736626320566707]
  We develop a prescriptive model of multi-agent behavior using Markov games. We focus on models in which the agents play "exploration but near optimum strategies"
  arXiv  Detail & Related papers  (2020-05-25T19:34:29Z)
• Distributed Reinforcement Learning for Cooperative Multi-Robot Object Manipulation [53.262360083572005]
  We consider solving a cooperative multi-robot object manipulation task using reinforcement learning (RL) We propose two distributed multi-agent RL approaches: distributed approximate RL (DA-RL) and game-theoretic RL (GT-RL) Although we focus on a small system of two agents in this paper, both DA-RL and GT-RL apply to general multi-agent systems, and are expected to scale well to large systems.
  arXiv  Detail & Related papers  (2020-03-21T00:43:54Z)
• Scalable Multi-Agent Inverse Reinforcement Learning via Actor-Attention-Critic [54.2180984002807]
  Multi-agent adversarial inverse reinforcement learning (MA-AIRL) is a recent approach that applies single-agent AIRL to multi-agent problems. We propose a multi-agent inverse RL algorithm that is more sample-efficient and scalable than previous works.
  arXiv  Detail & Related papers  (2020-02-24T20:30:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.