The Evolutionary Dynamics of Independent Learning Agents in Population Games

• URL: http://arxiv.org/abs/2006.16068v1
• Date: Mon, 29 Jun 2020 14:22:23 GMT
• Title: The Evolutionary Dynamics of Independent Learning Agents in Population Games
• Authors: Shuyue Hu, Chin-Wing Leung, Ho-fung Leung, Harold Soh
• Abstract summary: This paper presents a formal relation between processes and the dynamics of independent learning agents in population games. Using a master equation approach, we provide a novel unified framework for characterising population dynamics.
• Score: 21.68881173635777
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding the evolutionary dynamics of reinforcement learning under multi-agent settings has long remained an open problem. While previous works primarily focus on 2-player games, we consider population games, which model the strategic interactions of a large population comprising small and anonymous agents. This paper presents a formal relation between stochastic processes and the dynamics of independent learning agents who reason based on the reward signals. Using a master equation approach, we provide a novel unified framework for characterising population dynamics via a single partial differential equation (Theorem 1). Through a case study involving Cross learning agents, we illustrate that Theorem 1 allows us to identify qualitatively different evolutionary dynamics, to analyse steady states, and to gain insights into the expected behaviour of a population. In addition, we present extensive experimental results validating that Theorem 1 holds for a variety of learning methods and population games.

Related papers

• Evolution with Opponent-Learning Awareness [10.689403855269704]
  We show how large heterogeneous populations of learning agents evolve in normal-form games. We derive a fast, parallelizable implementation of Opponent-Learning Awareness tailored for evolutionary simulations. We demonstrate our approach in simulations of 200,000 agents, evolving in the classic games of Hawk-Dove, Stag-Hunt, and Rock-Paper-Scissors.
  arXiv  Detail & Related papers  (2024-10-22T22:49:04Z)
• PersLLM: A Personified Training Approach for Large Language Models [66.16513246245401]
  We propose PersLLM, integrating psychology-grounded principles of personality: social practice, consistency, and dynamic development. We incorporate personality traits directly into the model parameters, enhancing the model's resistance to induction, promoting consistency, and supporting the dynamic evolution of personality.
  arXiv  Detail & Related papers  (2024-07-17T08:13:22Z)
• Mathematics of multi-agent learning systems at the interface of game theory and artificial intelligence [0.8049333067399385]
  Evolutionary Game Theory and Artificial Intelligence are two fields that, at first glance, might seem distinct, but they have notable connections and intersections. The former focuses on the evolution of behaviors (or strategies) in a population, where individuals interact with others and update their strategies based on imitation (or social learning) The latter, meanwhile, is centered on machine learning algorithms and (deep) neural networks.
  arXiv  Detail & Related papers  (2024-03-09T17:36:54Z)
• DARLEI: Deep Accelerated Reinforcement Learning with Evolutionary Intelligence [77.78795329701367]
  We present DARLEI, a framework that combines evolutionary algorithms with parallelized reinforcement learning. We characterize DARLEI's performance under various conditions, revealing factors impacting diversity of evolved morphologies. We hope to extend DARLEI in future work to include interactions between diverse morphologies in richer environments.
  arXiv  Detail & Related papers  (2023-12-08T16:51:10Z)
• Interpretable Imitation Learning with Dynamic Causal Relations [65.18456572421702]
  We propose to expose captured knowledge in the form of a directed acyclic causal graph. We also design this causal discovery process to be state-dependent, enabling it to model the dynamics in latent causal graphs. The proposed framework is composed of three parts: a dynamic causal discovery module, a causality encoding module, and a prediction module, and is trained in an end-to-end manner.
  arXiv  Detail & Related papers  (2023-09-30T20:59:42Z)
• Decentralized Adversarial Training over Graphs [55.28669771020857]
  The vulnerability of machine learning models to adversarial attacks has been attracting considerable attention in recent years. This work studies adversarial training over graphs, where individual agents are subjected to varied strength perturbation space.
  arXiv  Detail & Related papers  (2023-03-23T15:05:16Z)
• Learning signatures of decision making from many individuals playing the same game [54.33783158658077]
  We design a predictive framework that learns representations to encode an individual's 'behavioral style' We apply our method to a large-scale behavioral dataset from 1,000 humans playing a 3-armed bandit task.
  arXiv  Detail & Related papers  (2023-02-21T21:41:53Z)
• Independent Learning in Stochastic Games [16.505046191280634]
  We present the model of games for multi-agent learning in dynamic environments. We focus on the development of simple and independent learning dynamics for games. We present our recently proposed simple and independent learning dynamics that guarantee convergence in zero-sum games.
  arXiv  Detail & Related papers  (2021-11-23T09:27:20Z)
• Multi-Agent Imitation Learning with Copulas [102.27052968901894]
  Multi-agent imitation learning aims to train multiple agents to perform tasks from demonstrations by learning a mapping between observations and actions. In this paper, we propose to use copula, a powerful statistical tool for capturing dependence among random variables, to explicitly model the correlation and coordination in multi-agent systems. Our proposed model is able to separately learn marginals that capture the local behavioral patterns of each individual agent, as well as a copula function that solely and fully captures the dependence structure among agents.
  arXiv  Detail & Related papers  (2021-07-10T03:49:41Z)
• Evolutionary Strategies with Analogy Partitions in p-guessing Games [0.0]
  We introduce an evolutionary process of learning to investigate the dynamics of learning in unstable p-guessing games environments. We show that our genetic algorithm behaves consistently with previous results in persistent environments, converging to the Nash equilibrium.
  arXiv  Detail & Related papers  (2021-03-26T10:28:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.