Learning Latent Representations to Influence Multi-Agent Interaction

• URL: http://arxiv.org/abs/2011.06619v1
• Date: Thu, 12 Nov 2020 19:04:26 GMT
• Title: Learning Latent Representations to Influence Multi-Agent Interaction
• Authors: Annie Xie, Dylan P. Losey, Ryan Tolsma, Chelsea Finn, Dorsa Sadigh
• Abstract summary: We propose a reinforcement learning-based framework for learning latent representations of an agent's policy. We show that our approach outperforms the alternatives and learns to influence the other agent.
• Score: 65.44092264843538
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Seamlessly interacting with humans or robots is hard because these agents are non-stationary. They update their policy in response to the ego agent's behavior, and the ego agent must anticipate these changes to co-adapt. Inspired by humans, we recognize that robots do not need to explicitly model every low-level action another agent will make; instead, we can capture the latent strategy of other agents through high-level representations. We propose a reinforcement learning-based framework for learning latent representations of an agent's policy, where the ego agent identifies the relationship between its behavior and the other agent's future strategy. The ego agent then leverages these latent dynamics to influence the other agent, purposely guiding them towards policies suitable for co-adaptation. Across several simulated domains and a real-world air hockey game, our approach outperforms the alternatives and learns to influence the other agent.

Related papers

• Contrastive learning-based agent modeling for deep reinforcement learning [31.293496061727932]
  Agent modeling is essential when designing adaptive policies for intelligent machine agents in multiagent systems. We devised a Contrastive Learning-based Agent Modeling (CLAM) method that relies only on the local observations from the ego agent during training and execution. CLAM is capable of generating consistent high-quality policy representations in real-time right from the beginning of each episode.
  arXiv  Detail & Related papers  (2023-12-30T03:44:12Z)
• DCIR: Dynamic Consistency Intrinsic Reward for Multi-Agent Reinforcement Learning [84.22561239481901]
  We propose a new approach that enables agents to learn whether their behaviors should be consistent with that of other agents. We evaluate DCIR in multiple environments including Multi-agent Particle, Google Research Football and StarCraft II Micromanagement.
  arXiv  Detail & Related papers  (2023-12-10T06:03:57Z)
• ProAgent: Building Proactive Cooperative Agents with Large Language Models [89.53040828210945]
  ProAgent is a novel framework that harnesses large language models to create proactive agents. ProAgent can analyze the present state, and infer the intentions of teammates from observations. ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various coordination scenarios.
  arXiv  Detail & Related papers  (2023-08-22T10:36:56Z)
• Diversifying Agent's Behaviors in Interactive Decision Models [11.125175635860169]
  Modelling other agents' behaviors plays an important role in decision models for interactions among multiple agents. In this article, we investigate into diversifying behaviors of other agents in the subject agent's decision model prior to their interactions.
  arXiv  Detail & Related papers  (2022-03-06T23:05:00Z)
• Influencing Towards Stable Multi-Agent Interactions [12.477674452685756]
  Learning in multi-agent environments is difficult due to the non-stationarity introduced by an opponent's or partner's changing behaviors. We propose an algorithm to proactively influence the other agent's strategy to stabilize. We demonstrate the effectiveness of stabilizing in improving efficiency of maximizing the task reward in a variety of simulated environments.
  arXiv  Detail & Related papers  (2021-10-05T16:46:04Z)
• Deep Interactive Bayesian Reinforcement Learning via Meta-Learning [63.96201773395921]
  The optimal adaptive behaviour under uncertainty over the other agents' strategies can be computed using the Interactive Bayesian Reinforcement Learning framework. We propose to meta-learn approximate belief inference and Bayes-optimal behaviour for a given prior. We show empirically that our approach outperforms existing methods that use a model-free approach, sample from the approximate posterior, maintain memory-free models of others, or do not fully utilise the known structure of the environment.
  arXiv  Detail & Related papers  (2021-01-11T13:25:13Z)
• A Policy Gradient Algorithm for Learning to Learn in Multiagent Reinforcement Learning [47.154539984501895]
  We propose a novel meta-multiagent policy gradient theorem that accounts for the non-stationary policy dynamics inherent to multiagent learning settings. This is achieved by modeling our gradient updates to consider both an agent's own non-stationary policy dynamics and the non-stationary policy dynamics of other agents in the environment.
  arXiv  Detail & Related papers  (2020-10-31T22:50:21Z)
• Learning to Incentivize Other Learning Agents [73.03133692589532]
  We show how to equip RL agents with the ability to give rewards directly to other agents, using a learned incentive function. Such agents significantly outperform standard RL and opponent-shaping agents in challenging general-sum Markov games. Our work points toward more opportunities and challenges along the path to ensure the common good in a multi-agent future.
  arXiv  Detail & Related papers  (2020-06-10T20:12:38Z)
• Variational Autoencoders for Opponent Modeling in Multi-Agent Systems [9.405879323049659]
  Multi-agent systems exhibit complex behaviors that emanate from the interactions of multiple agents in a shared environment. In this work, we are interested in controlling one agent in a multi-agent system and successfully learn to interact with the other agents that have fixed policies. Modeling the behavior of other agents (opponents) is essential in understanding the interactions of the agents in the system.
  arXiv  Detail & Related papers  (2020-01-29T13:38:59Z)
• Towards Learning Multi-agent Negotiations via Self-Play [2.28438857884398]
  We show how an iterative procedure of self-play can create progressively more diverse environments. This leads to the learning of sophisticated and robust multi-agent policies. We show a dramatic improvement in the success rate of merging maneuvers from 63% to over 98%.
  arXiv  Detail & Related papers  (2020-01-28T08:37:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.