Moody Learners -- Explaining Competitive Behaviour of Reinforcement Learning Agents

• URL: http://arxiv.org/abs/2007.16045v1
• Date: Thu, 30 Jul 2020 11:30:42 GMT
• Title: Moody Learners -- Explaining Competitive Behaviour of Reinforcement Learning Agents
• Authors: Pablo Barros, Ana Tanevska, Francisco Cruz, Alessandra Sciutti
• Abstract summary: In a competitive scenario, the agent does not only have a dynamic environment but also is directly affected by the opponents' actions. Observing the Q-values of the agent is usually a way of explaining its behavior, however, do not show the temporal-relation between the selected actions.
• Score: 65.2200847818153
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Designing the decision-making processes of artificial agents that are involved in competitive interactions is a challenging task. In a competitive scenario, the agent does not only have a dynamic environment but also is directly affected by the opponents' actions. Observing the Q-values of the agent is usually a way of explaining its behavior, however, do not show the temporal-relation between the selected actions. We address this problem by proposing the \emph{Moody framework}. We evaluate our model by performing a series of experiments using the competitive multiplayer Chef's Hat card game and discuss how our model allows the agents' to obtain a holistic representation of the competitive dynamics within the game.

Related papers

• Toward Optimal LLM Alignments Using Two-Player Games [86.39338084862324]
  In this paper, we investigate alignment through the lens of two-agent games, involving iterative interactions between an adversarial and a defensive agent. We theoretically demonstrate that this iterative reinforcement learning optimization converges to a Nash Equilibrium for the game induced by the agents. Experimental results in safety scenarios demonstrate that learning in such a competitive environment not only fully trains agents but also leads to policies with enhanced generalization capabilities for both adversarial and defensive agents.
  arXiv  Detail & Related papers  (2024-06-16T15:24:50Z)
• ProAgent: Building Proactive Cooperative Agents with Large Language Models [89.53040828210945]
  ProAgent is a novel framework that harnesses large language models to create proactive agents. ProAgent can analyze the present state, and infer the intentions of teammates from observations. ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various coordination scenarios.
  arXiv  Detail & Related papers  (2023-08-22T10:36:56Z)
• Decision-making with Speculative Opponent Models [10.594910251058087]
  We introduce Distributional Opponent-aided Multi-agent Actor-Critic (DOMAC) DOMAC is the first speculative opponent modelling algorithm that relies solely on local information (i.e., the controlled agent's observations, actions, and rewards)
  arXiv  Detail & Related papers  (2022-11-22T01:29:47Z)
• Game-theoretic Objective Space Planning [4.989480853499916]
  Understanding intent of other agents is crucial to deploying autonomous systems in adversarial multi-agent environments. Current approaches either oversimplify the discretization of the action space of agents or fail to recognize the long-term effect of actions and become myopic. We propose a novel dimension reduction method that encapsulates diverse agent behaviors while conserving the continuity of agent actions.
  arXiv  Detail & Related papers  (2022-09-16T07:35:20Z)
• Incorporating Rivalry in Reinforcement Learning for a Competitive Game [65.2200847818153]
  This work proposes a novel reinforcement learning mechanism based on the social impact of rivalry behavior. Our proposed model aggregates objective and social perception mechanisms to derive a rivalry score that is used to modulate the learning of artificial agents.
  arXiv  Detail & Related papers  (2022-08-22T14:06:06Z)
• Multi-agent Actor-Critic with Time Dynamical Opponent Model [16.820873906787906]
  In multi-agent reinforcement learning, multiple agents learn simultaneously while interacting with a common environment and each other. We propose a novel textitTime Dynamical Opponent Model (TDOM) to encode the knowledge that the opponent policies tend to improve over time. We show empirically that TDOM achieves superior opponent behavior prediction during test time.
  arXiv  Detail & Related papers  (2022-04-12T07:16:15Z)
• Learning Latent Representations to Influence Multi-Agent Interaction [65.44092264843538]
  We propose a reinforcement learning-based framework for learning latent representations of an agent's policy. We show that our approach outperforms the alternatives and learns to influence the other agent.
  arXiv  Detail & Related papers  (2020-11-12T19:04:26Z)
• Incorporating Rivalry in Reinforcement Learning for a Competitive Game [65.2200847818153]
  This study focuses on providing a novel learning mechanism based on a rivalry social impact. Based on the concept of competitive rivalry, our analysis aims to investigate if we can change the assessment of these agents from a human perspective.
  arXiv  Detail & Related papers  (2020-11-02T21:54:18Z)
• Learning to Model Opponent Learning [11.61673411387596]
  Multi-Agent Reinforcement Learning (MARL) considers settings in which a set of coexisting agents interact with one another and their environment. This poses a great challenge for value function-based algorithms whose convergence usually relies on the assumption of a stationary environment. We develop a novel approach to modelling an opponent's learning dynamics which we term Learning to Model Opponent Learning (LeMOL)
  arXiv  Detail & Related papers  (2020-06-06T17:19:04Z)
• Variational Autoencoders for Opponent Modeling in Multi-Agent Systems [9.405879323049659]
  Multi-agent systems exhibit complex behaviors that emanate from the interactions of multiple agents in a shared environment. In this work, we are interested in controlling one agent in a multi-agent system and successfully learn to interact with the other agents that have fixed policies. Modeling the behavior of other agents (opponents) is essential in understanding the interactions of the agents in the system.
  arXiv  Detail & Related papers  (2020-01-29T13:38:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.