Paying to Do Better: Games with Payments between Learning Agents

• URL: http://arxiv.org/abs/2405.20880v2
• Date: Tue, 11 Feb 2025 16:29:04 GMT
• Title: Paying to Do Better: Games with Payments between Learning Agents
• Authors: Yoav Kolumbus, Joe Halpern, Éva Tardos,
• Abstract summary: We study the impact of players incorporating monetary transfer policies into their learning agents' algorithms. We propose a simple and general game-theoretic model to capture such scenarios. Results show that in a very broad class of games, self-interested players benefit from letting their learning agents make payments to other learners.
• Score: 4.067193517689939
• License:
• Abstract: In repeated games, such as auctions, players typically use learning algorithms to choose their actions. The use of such autonomous learning agents has become widespread on online platforms. In this paper, we explore the impact of players incorporating monetary transfer policies into their agents' algorithms, aiming to influence behavior in their favor through the dynamics between the agents. Our focus is on understanding when players have incentives to make use of monetary transfers, how such payments may affect learning dynamics, and what the implications are for welfare and its distribution among the players. We propose a simple and general game-theoretic model to capture such scenarios. Our results on general games show that in a very broad class of games, self-interested players benefit from letting their learning agents make payments to other learners during the game dynamics, and that in many cases, this kind of behavior improves welfare for all players. Our results on first- and second-price auctions show that in equilibria of the ``payment policy game,'' the agents' dynamics reach strong collusive outcomes with low revenue for the auctioneer. These results raise new questions and highlight a challenge for mechanism design in systems where automated learning agents can benefit from interacting with their peers in the digital ecosystem and outside the boundaries of the mechanism.

Related papers

• Expectation vs. Reality: Towards Verification of Psychological Games [18.30789345402813]
  Psychological games (PGs) were developed as a way to model and analyse agents with belief-dependent motivations. This paper proposes methods to solve PGs and implementing them within PRISM-games, a formal verification tool for games.
  arXiv  Detail & Related papers  (2024-11-08T14:41:52Z)
• Reciprocal Reward Influence Encourages Cooperation From Self-Interested Agents [2.1301560294088318]
  Cooperation between self-interested individuals is a widespread phenomenon in the natural world, but remains elusive in interactions between artificially intelligent agents. We introduce Reciprocators, reinforcement learning agents which are intrinsically motivated to reciprocate the influence of opponents' actions on their returns. We show that Reciprocators can be used to promote cooperation in temporally extended social dilemmas during simultaneous learning.
  arXiv  Detail & Related papers  (2024-06-03T06:07:27Z)
• Incentivized Learning in Principal-Agent Bandit Games [62.41639598376539]
  This work considers a repeated principal-agent bandit game, where the principal can only interact with her environment through the agent. The principal can influence the agent's decisions by offering incentives which add up to his rewards. We present nearly optimal learning algorithms for the principal's regret in both multi-armed and linear contextual settings.
  arXiv  Detail & Related papers  (2024-03-06T16:00:46Z)
• Impact of Decentralized Learning on Player Utilities in Stackelberg Games [57.08270857260131]
  In many two-agent systems, each agent learns separately and the rewards of the two agents are not perfectly aligned. We model these systems as Stackelberg games with decentralized learning and show that standard regret benchmarks result in worst-case linear regret for at least one player. We develop algorithms to achieve near-optimal $O(T2/3)$ regret for both players with respect to these benchmarks.
  arXiv  Detail & Related papers  (2024-02-29T23:38:28Z)
• How Bad is Selfish Driving? Bounding the Inefficiency of Equilibria in Urban Driving Games [64.71476526716668]
  We study the (in)efficiency of any equilibrium players might agree to play. We obtain guarantees that refine existing bounds on the Price of Anarchy. Although the obtained guarantees concern open-loop trajectories, we observe efficient equilibria even when agents employ closed-loop policies.
  arXiv  Detail & Related papers  (2022-10-24T09:32:40Z)
• Incorporating Rivalry in Reinforcement Learning for a Competitive Game [65.2200847818153]
  This work proposes a novel reinforcement learning mechanism based on the social impact of rivalry behavior. Our proposed model aggregates objective and social perception mechanisms to derive a rivalry score that is used to modulate the learning of artificial agents.
  arXiv  Detail & Related papers  (2022-08-22T14:06:06Z)
• How and Why to Manipulate Your Own Agent [5.634825161148484]
  We consider strategic settings where several users engage in a repeated online interaction, assisted by regret-minimizing agents that repeatedly play a "game" on their behalf. We study the dynamics and average outcomes of the repeated game of the agents, and view it as inducing a meta-game between the users.
  arXiv  Detail & Related papers  (2021-12-14T18:35:32Z)
• Player Modeling using Behavioral Signals in Competitive Online Games [4.168733556014873]
  This paper focuses on the importance of addressing different aspects of playing behavior when modeling players for creating match-ups. We engineer several behavioral features from a dataset of over 75,000 battle royale matches and create player models. We then use the created models to predict ranks for different groups of players in the data.
  arXiv  Detail & Related papers  (2021-11-29T22:53:17Z)
• PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning [102.36450942613091]
  We propose an inverse reinforcement learning algorithm, called emphinverse temporal difference learning (ITD) We show how to seamlessly integrate ITD with learning from online environment interactions, arriving at a novel algorithm for reinforcement learning with demonstrations, called $Psi Phi$-learning.
  arXiv  Detail & Related papers  (2021-02-24T21:12:09Z)
• Learning from Learners: Adapting Reinforcement Learning Agents to be Competitive in a Card Game [71.24825724518847]
  We present a study on how popular reinforcement learning algorithms can be adapted to learn and to play a real-world implementation of a competitive multiplayer card game. We propose specific training and validation routines for the learning agents, in order to evaluate how the agents learn to be competitive and explain how they adapt to each others' playing style.
  arXiv  Detail & Related papers  (2020-04-08T14:11:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.