Reinforcement Learning, Collusion, and the Folk Theorem

• URL: http://arxiv.org/abs/2411.12725v1
• Date: Tue, 19 Nov 2024 18:45:55 GMT
• Title: Reinforcement Learning, Collusion, and the Folk Theorem
• Authors: Galit Askenazi-Golan, Domenico Mergoni Cecchelli, Edward Plumb,
• Abstract summary: We explore the behaviour emerging from learning agents repeatedly interacting strategically for a wide range of learning dynamics. We consider the setting of a general repeated game with finite recall, for different forms of monitoring. We obtain a Folk Theorem-like result and characterise the set of payoff vectors that can be obtained by these dynamics.
• Score: 0.0
• License:
• Abstract: We explore the behaviour emerging from learning agents repeatedly interacting strategically for a wide range of learning dynamics that includes projected gradient, replicator and log-barrier dynamics. Going beyond the better-understood classes of potential games and zero-sum games, we consider the setting of a general repeated game with finite recall, for different forms of monitoring. We obtain a Folk Theorem-like result and characterise the set of payoff vectors that can be obtained by these dynamics, discovering a wide range of possibilities for the emergence of algorithmic collusion.

Related papers

• Nested replicator dynamics, nested logit choice, and similarity-based learning [56.98352103321524]
  We consider a model of learning and evolution in games with action sets endowed with a partition-based similarity structure. In this model, revising agents have a higher probability of comparing their current strategy with other strategies that they deem similar. Because of this implicit bias toward similar strategies, the resulting dynamics do not satisfy any of the standard monotonicity rationalitys for imitative game dynamics.
  arXiv  Detail & Related papers  (2024-07-25T07:09:53Z)
• A unified stochastic approximation framework for learning in games [82.74514886461257]
  We develop a flexible approximation framework for analyzing the long-run behavior of learning in games (both continuous and finite) The proposed analysis template incorporates a wide array of popular learning algorithms, including gradient-based methods, exponential/multiplicative weights for learning in finite games, optimistic and bandit variants of the above, etc.
  arXiv  Detail & Related papers  (2022-06-08T14:30:38Z)
• Adaptive Discrete Communication Bottlenecks with Dynamic Vector Quantization [76.68866368409216]
  We propose learning to dynamically select discretization tightness conditioned on inputs. We show that dynamically varying tightness in communication bottlenecks can improve model performance on visual reasoning and reinforcement learning tasks.
  arXiv  Detail & Related papers  (2022-02-02T23:54:26Z)
• Learning Generalizable Behavior via Visual Rewrite Rules [0.9558392439655015]
  In this paper, we propose a novel representation and learning approach to capture environment dynamics without using neural networks. It originates from the observation that, in games designed for people, the effect of an action can often be perceived in the form of local changes in consecutive visual observations. Our algorithm is designed to extract such vision-based changes and condense them into a set of action-dependent descriptive rules, which we call ''visual rewrite rules'' (VRRs)
  arXiv  Detail & Related papers  (2021-12-09T21:23:26Z)
• Independent Learning in Stochastic Games [16.505046191280634]
  We present the model of games for multi-agent learning in dynamic environments. We focus on the development of simple and independent learning dynamics for games. We present our recently proposed simple and independent learning dynamics that guarantee convergence in zero-sum games.
  arXiv  Detail & Related papers  (2021-11-23T09:27:20Z)
• Generative Adversarial Reward Learning for Generalized Behavior Tendency Inference [71.11416263370823]
  We propose a generative inverse reinforcement learning for user behavioral preference modelling. Our model can automatically learn the rewards from user's actions based on discriminative actor-critic network and Wasserstein GAN.
  arXiv  Detail & Related papers  (2021-05-03T13:14:25Z)
• Opponent Learning Awareness and Modelling in Multi-Objective Normal Form Games [5.0238343960165155]
  It is essential for an agent to learn about the behaviour of other agents in the system. We present the first study of the effects of such opponent modelling on multi-objective multi-agent interactions with non-linear utilities.
  arXiv  Detail & Related papers  (2020-11-14T12:35:32Z)
• Behavior Priors for Efficient Reinforcement Learning [97.81587970962232]
  We consider how information and architectural constraints can be combined with ideas from the probabilistic modeling literature to learn behavior priors. We discuss how such latent variable formulations connect to related work on hierarchical reinforcement learning (HRL) and mutual information and curiosity based objectives. We demonstrate the effectiveness of our framework by applying it to a range of simulated continuous control domains.
  arXiv  Detail & Related papers  (2020-10-27T13:17:18Z)
• The Evolutionary Dynamics of Independent Learning Agents in Population Games [21.68881173635777]
  This paper presents a formal relation between processes and the dynamics of independent learning agents in population games. Using a master equation approach, we provide a novel unified framework for characterising population dynamics.
  arXiv  Detail & Related papers  (2020-06-29T14:22:23Z)
• Guided Variational Autoencoder for Disentanglement Learning [79.02010588207416]
  We propose an algorithm, guided variational autoencoder (Guided-VAE), that is able to learn a controllable generative model by performing latent representation disentanglement learning. We design an unsupervised strategy and a supervised strategy in Guided-VAE and observe enhanced modeling and controlling capability over the vanilla VAE.
  arXiv  Detail & Related papers  (2020-04-02T20:49:15Z)
• Distributed No-Regret Learning in Multi-Agent Systems [12.111429383532888]
  Four emerging game characteristics challenge canonical game models are explored. For each of the four characteristics, we illuminate its implications and ramifications in game modeling, notions of regret, feasible game outcomes, and the design and analysis of distributed learning algorithms.
  arXiv  Detail & Related papers  (2020-02-20T22:30:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.