Influencing Towards Stable Multi-Agent Interactions

• URL: http://arxiv.org/abs/2110.08229v1
• Date: Tue, 5 Oct 2021 16:46:04 GMT
• Title: Influencing Towards Stable Multi-Agent Interactions
• Authors: Woodrow Z. Wang, Andy Shih, Annie Xie, Dorsa Sadigh
• Abstract summary: Learning in multi-agent environments is difficult due to the non-stationarity introduced by an opponent's or partner's changing behaviors. We propose an algorithm to proactively influence the other agent's strategy to stabilize. We demonstrate the effectiveness of stabilizing in improving efficiency of maximizing the task reward in a variety of simulated environments.
• Score: 12.477674452685756
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning in multi-agent environments is difficult due to the non-stationarity introduced by an opponent's or partner's changing behaviors. Instead of reactively adapting to the other agent's (opponent or partner) behavior, we propose an algorithm to proactively influence the other agent's strategy to stabilize -- which can restrain the non-stationarity caused by the other agent. We learn a low-dimensional latent representation of the other agent's strategy and the dynamics of how the latent strategy evolves with respect to our robot's behavior. With this learned dynamics model, we can define an unsupervised stability reward to train our robot to deliberately influence the other agent to stabilize towards a single strategy. We demonstrate the effectiveness of stabilizing in improving efficiency of maximizing the task reward in a variety of simulated environments, including autonomous driving, emergent communication, and robotic manipulation. We show qualitative results on our website: https://sites.google.com/view/stable-marl/.

Related papers

• Learning responsibility allocations for multi-agent interactions: A differentiable optimization approach with control barrier functions [12.074590482085831]
  We seek to codify factors governing safe multi-agent interactions via the lens of responsibility. We propose a data-driven modeling approach based on control barrier functions and differentiable optimization.
  arXiv  Detail & Related papers  (2024-10-09T20:20:41Z)
• DCIR: Dynamic Consistency Intrinsic Reward for Multi-Agent Reinforcement Learning [84.22561239481901]
  We propose a new approach that enables agents to learn whether their behaviors should be consistent with that of other agents. We evaluate DCIR in multiple environments including Multi-agent Particle, Google Research Football and StarCraft II Micromanagement.
  arXiv  Detail & Related papers  (2023-12-10T06:03:57Z)
• Learning Goal-based Movement via Motivational-based Models in Cognitive Mobile Robots [58.720142291102135]
  Humans have needs motivating their behavior according to intensity and context. We also create preferences associated with each action's perceived pleasure, which is susceptible to changes over time. This makes decision-making more complex, requiring learning to balance needs and preferences according to the context.
  arXiv  Detail & Related papers  (2023-02-20T04:52:24Z)
• Coach-assisted Multi-Agent Reinforcement Learning Framework for Unexpected Crashed Agents [120.91291581594773]
  We present a formal formulation of a cooperative multi-agent reinforcement learning system with unexpected crashes. We propose a coach-assisted multi-agent reinforcement learning framework, which introduces a virtual coach agent to adjust the crash rate during training. To the best of our knowledge, this work is the first to study the unexpected crashes in the multi-agent system.
  arXiv  Detail & Related papers  (2022-03-16T08:22:45Z)
• ROMAX: Certifiably Robust Deep Multiagent Reinforcement Learning via Convex Relaxation [32.091346776897744]
  Cyber-physical attacks can challenge the robustness of multiagent reinforcement learning. We propose a minimax MARL approach to infer the worst-case policy update of other agents.
  arXiv  Detail & Related papers  (2021-09-14T16:18:35Z)
• Deep Interactive Bayesian Reinforcement Learning via Meta-Learning [63.96201773395921]
  The optimal adaptive behaviour under uncertainty over the other agents' strategies can be computed using the Interactive Bayesian Reinforcement Learning framework. We propose to meta-learn approximate belief inference and Bayes-optimal behaviour for a given prior. We show empirically that our approach outperforms existing methods that use a model-free approach, sample from the approximate posterior, maintain memory-free models of others, or do not fully utilise the known structure of the environment.
  arXiv  Detail & Related papers  (2021-01-11T13:25:13Z)
• Learning Latent Representations to Influence Multi-Agent Interaction [65.44092264843538]
  We propose a reinforcement learning-based framework for learning latent representations of an agent's policy. We show that our approach outperforms the alternatives and learns to influence the other agent.
  arXiv  Detail & Related papers  (2020-11-12T19:04:26Z)
• MIDAS: Multi-agent Interaction-aware Decision-making with Adaptive Strategies for Urban Autonomous Navigation [22.594295184455]
  This paper builds a reinforcement learning-based method named MIDAS where an ego-agent learns to affect the control actions of other cars. MIDAS is validated using extensive experiments and we show that it (i) can work across different road geometries, (ii) is robust to changes in the driving policies of external agents, and (iv) is more efficient and safer than existing approaches to interaction-aware decision-making.
  arXiv  Detail & Related papers  (2020-08-17T04:34:25Z)
• Efficient Empowerment Estimation for Unsupervised Stabilization [75.32013242448151]
  empowerment principle enables unsupervised stabilization of dynamical systems at upright positions. We propose an alternative solution based on a trainable representation of a dynamical system as a Gaussian channel. We show that our method has a lower sample complexity, is more stable in training, possesses the essential properties of the empowerment function, and allows estimation of empowerment from images.
  arXiv  Detail & Related papers  (2020-07-14T21:10:16Z)
• Towards Learning Multi-agent Negotiations via Self-Play [2.28438857884398]
  We show how an iterative procedure of self-play can create progressively more diverse environments. This leads to the learning of sophisticated and robust multi-agent policies. We show a dramatic improvement in the success rate of merging maneuvers from 63% to over 98%.
  arXiv  Detail & Related papers  (2020-01-28T08:37:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.