Winning at Any Cost -- Infringing the Cartel Prohibition With Reinforcement Learning

• URL: http://arxiv.org/abs/2107.01856v1
• Date: Mon, 5 Jul 2021 08:21:52 GMT
• Title: Winning at Any Cost -- Infringing the Cartel Prohibition With Reinforcement Learning
• Authors: Michael Schlechtinger, Damaris Kosack, Heiko Paulheim, Thomas Fetzer
• Abstract summary: In e-commerce scenarios, multiple reinforcement learning agents can set prices based on their competitor's prices. We build a scenario that is based on a modified version of a prisoner's dilemma where three agents play the game of rock paper scissors. Our results indicate that the action selection can be dissected into specific stages, establishing the possibility to develop collusion prevention systems.
• Score: 1.1470070927586016
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Pricing decisions are increasingly made by AI. Thanks to their ability to train with live market data while making decisions on the fly, deep reinforcement learning algorithms are especially effective in taking such pricing decisions. In e-commerce scenarios, multiple reinforcement learning agents can set prices based on their competitor's prices. Therefore, research states that agents might end up in a state of collusion in the long run. To further analyze this issue, we build a scenario that is based on a modified version of a prisoner's dilemma where three agents play the game of rock paper scissors. Our results indicate that the action selection can be dissected into specific stages, establishing the possibility to develop collusion prevention systems that are able to recognize situations which might lead to a collusion between competitors. We furthermore provide evidence for a situation where agents are capable of performing a tacit cooperation strategy without being explicitly trained to do so.

Related papers

• Naive Algorithmic Collusion: When Do Bandit Learners Cooperate and When Do They Compete? [0.0]
  Algorithmic agents are used in a variety of competitive decision settings. We study the behavior of multi-armed bandit machine learning algorithms used in situations where agents are competing. We show that these context-free bandits, with no knowledge of opponents' choices or outcomes, still will consistently learn collusive behavior.
  arXiv  Detail & Related papers  (2024-11-25T16:58:07Z)
• Artificial Intelligence and Algorithmic Price Collusion in Two-sided Markets [9.053163124987535]
  We examine how AI agents using Q-learning engage in tacit collusion in two-sided markets. Our experiments reveal that AI-driven platforms achieve higher collusion levels compared to Bertrand competition. Increased network externalities significantly enhance collusion, suggesting AI algorithms exploit them to maximize profits.
  arXiv  Detail & Related papers  (2024-07-04T17:57:56Z)
• Tacit algorithmic collusion in deep reinforcement learning guided price competition: A study using EV charge pricing game [0.0]
  Players in pricing games with complex structures are increasingly adopting artificial intelligence (AI) aided learning algorithms. Recent studies of games in canonical forms have shown contrasting claims ranging from none to a high level of tacit collusion. We consider a practical game where EV charging hubs compete by dynamically varying their prices. Results from our numerical case study yield collusion index values between 0.14 and 0.45, suggesting a low to moderate level of collusion.
  arXiv  Detail & Related papers  (2024-01-25T16:51:52Z)
• Designing an attack-defense game: how to increase robustness of financial transaction models via a competition [69.08339915577206]
  Given the escalating risks of malicious attacks in the finance sector, understanding adversarial strategies and robust defense mechanisms for machine learning models is critical. We aim to investigate the current state and dynamics of adversarial attacks and defenses for neural network models that use sequential financial data as the input. We have designed a competition that allows realistic and detailed investigation of problems in modern financial transaction data. The participants compete directly against each other, so possible attacks and defenses are examined in close-to-real-life conditions.
  arXiv  Detail & Related papers  (2023-08-22T12:53:09Z)
• Learning in Stackelberg Games with Non-myopic Agents [60.927889817803745]
  We study Stackelberg games where a principal repeatedly interacts with a non-myopic long-lived agent, without knowing the agent's payoff function. We provide a general framework that reduces learning in presence of non-myopic agents to robust bandit optimization in the presence of myopic agents.
  arXiv  Detail & Related papers  (2022-08-19T15:49:30Z)
• Stochastic Market Games [10.979093424231532]
  We propose to utilize market forces to provide incentives for agents to become cooperative. As demonstrated in an iterated version of the Prisoner's Dilemma, the proposed market formulation can change the dynamics of the game. We empirically find that the presence of markets can improve both the overall result and agent individual returns via their trading activities.
  arXiv  Detail & Related papers  (2022-07-15T10:37:16Z)
• Detecting adversaries in Crowdsourcing [71.20185379303479]
  This work investigates the effects of adversaries on crowdsourced classification, under the popular Dawid and Skene model. The adversaries are allowed to deviate arbitrarily from the considered crowdsourcing model, and may potentially cooperate. We develop an approach that leverages the structure of second-order moments of annotator responses, to identify large numbers of adversaries, and mitigate their impact on the crowdsourcing task.
  arXiv  Detail & Related papers  (2021-10-07T15:07:07Z)
• Moody Learners -- Explaining Competitive Behaviour of Reinforcement Learning Agents [65.2200847818153]
  In a competitive scenario, the agent does not only have a dynamic environment but also is directly affected by the opponents' actions. Observing the Q-values of the agent is usually a way of explaining its behavior, however, do not show the temporal-relation between the selected actions.
  arXiv  Detail & Related papers  (2020-07-30T11:30:42Z)
• Learning from Learners: Adapting Reinforcement Learning Agents to be Competitive in a Card Game [71.24825724518847]
  We present a study on how popular reinforcement learning algorithms can be adapted to learn and to play a real-world implementation of a competitive multiplayer card game. We propose specific training and validation routines for the learning agents, in order to evaluate how the agents learn to be competitive and explain how they adapt to each others' playing style.
  arXiv  Detail & Related papers  (2020-04-08T14:11:05Z)
• Multi-Issue Bargaining With Deep Reinforcement Learning [0.0]
  This paper evaluates the use of deep reinforcement learning in bargaining games. Two actor-critic networks were trained for the bidding and acceptance strategy. Neural agents learn to exploit time-based agents, achieving clear transitions in decision preference values. They also demonstrate adaptive behavior against different combinations of concession, discount factors, and behavior-based strategies.
  arXiv  Detail & Related papers  (2020-02-18T18:33:46Z)
• Adversarial Attacks on Linear Contextual Bandits [87.08004581867537]
  Malicious agents may have incentives to attack the bandit algorithm to induce it to perform a desired behavior. We show that a malicious agent can force a linear contextual bandit algorithm to pull any desired arm $T - o(T)$ times over a horizon of $T$ steps. We also investigate the case when a malicious agent is interested in affecting the behavior of the bandit algorithm in a single context.
  arXiv  Detail & Related papers  (2020-02-10T15:04:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.