Tacit algorithmic collusion in deep reinforcement learning guided price competition: A study using EV charge pricing game

• URL: http://arxiv.org/abs/2401.15108v2
• Date: Fri, 10 May 2024 10:24:12 GMT
• Title: Tacit algorithmic collusion in deep reinforcement learning guided price competition: A study using EV charge pricing game
• Authors: Diwas Paudel, Tapas K. Das,
• Abstract summary: Players in pricing games with complex structures are increasingly adopting artificial intelligence (AI) aided learning algorithms. Recent studies of games in canonical forms have shown contrasting claims ranging from none to a high level of tacit collusion. We consider a practical game where EV charging hubs compete by dynamically varying their prices. Results from our numerical case study yield collusion index values between 0.14 and 0.45, suggesting a low to moderate level of collusion.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Players in pricing games with complex structures are increasingly adopting artificial intelligence (AI) aided learning algorithms to make pricing decisions for maximizing profits. This is raising concern for the antitrust agencies as the practice of using AI may promote tacit algorithmic collusion among otherwise independent players. Recent studies of games in canonical forms have shown contrasting claims ranging from none to a high level of tacit collusion among AI-guided players. In this paper, we examine the concern for tacit collusion by considering a practical game where EV charging hubs compete by dynamically varying their prices. Such a game is likely to be commonplace in the near future as EV adoption grows in all sectors of transportation. The hubs source power from the day-ahead (DA) and real-time (RT) electricity markets as well as from in-house battery storage systems. Their goal is to maximize profits via pricing and efficiently managing the cost of power usage. To aid our examination, we develop a two-step data-driven methodology. The first step obtains the DA commitment by solving a stochastic model. The second step generates the pricing strategies by solving a competitive Markov decision process model using a multi-agent deep reinforcement learning (MADRL) framework. We evaluate the resulting pricing strategies using an index for the level of tacit algorithmic collusion. An index value of zero indicates no collusion (perfect competition) and one indicates full collusion (monopolistic behavior). Results from our numerical case study yield collusion index values between 0.14 and 0.45, suggesting a low to moderate level of collusion.

Related papers

• Naive Algorithmic Collusion: When Do Bandit Learners Cooperate and When Do They Compete? [0.0]
  Algorithmic agents are used in a variety of competitive decision settings. We study the behavior of multi-armed bandit machine learning algorithms used in situations where agents are competing. We show that these context-free bandits, with no knowledge of opponents' choices or outcomes, still will consistently learn collusive behavior.
  arXiv  Detail & Related papers  (2024-11-25T16:58:07Z)
• Artificial Intelligence and Algorithmic Price Collusion in Two-sided Markets [9.053163124987535]
  We examine how AI agents using Q-learning engage in tacit collusion in two-sided markets. Our experiments reveal that AI-driven platforms achieve higher collusion levels compared to Bertrand competition. Increased network externalities significantly enhance collusion, suggesting AI algorithms exploit them to maximize profits.
  arXiv  Detail & Related papers  (2024-07-04T17:57:56Z)
• By Fair Means or Foul: Quantifying Collusion in a Market Simulation with Deep Reinforcement Learning [1.5249435285717095]
  This research employs an experimental oligopoly model of repeated price competition. We investigate the strategies and emerging pricing patterns developed by the agents, which may lead to a collusive outcome. Our findings indicate that RL-based AI agents converge to a collusive state characterized by the charging of supracompetitive prices.
  arXiv  Detail & Related papers  (2024-06-04T15:35:08Z)
• Autobidders with Budget and ROI Constraints: Efficiency, Regret, and Pacing Dynamics [53.62091043347035]
  We study a game between autobidding algorithms that compete in an online advertising platform. We propose a gradient-based learning algorithm that is guaranteed to satisfy all constraints and achieves vanishing individual regret.
  arXiv  Detail & Related papers  (2023-01-30T21:59:30Z)
• Provably Efficient Fictitious Play Policy Optimization for Zero-Sum Markov Games with Structured Transitions [145.54544979467872]
  We propose and analyze new fictitious play policy optimization algorithms for zero-sum Markov games with structured but unknown transitions. We prove tight $widetildemathcalO(sqrtK)$ regret bounds after $K$ episodes in a two-agent competitive game scenario. Our algorithms feature a combination of Upper Confidence Bound (UCB)-type optimism and fictitious play under the scope of simultaneous policy optimization.
  arXiv  Detail & Related papers  (2022-07-25T18:29:16Z)
• Decentralized Optimistic Hyperpolicy Mirror Descent: Provably No-Regret Learning in Markov Games [95.10091348976779]
  We study decentralized policy learning in Markov games where we control a single agent to play with nonstationary and possibly adversarial opponents. We propose a new algorithm, underlineDecentralized underlineOptimistic hypeunderlineRpolicy munderlineIrror deunderlineScent (DORIS) DORIS achieves $sqrtK$-regret in the context of general function approximation, where $K$ is the number of episodes.
  arXiv  Detail & Related papers  (2022-06-03T14:18:05Z)
• Winning at Any Cost -- Infringing the Cartel Prohibition With Reinforcement Learning [1.1470070927586016]
  In e-commerce scenarios, multiple reinforcement learning agents can set prices based on their competitor's prices. We build a scenario that is based on a modified version of a prisoner's dilemma where three agents play the game of rock paper scissors. Our results indicate that the action selection can be dissected into specific stages, establishing the possibility to develop collusion prevention systems.
  arXiv  Detail & Related papers  (2021-07-05T08:21:52Z)
• Understanding algorithmic collusion with experience replay [0.0]
  In an infinitely repeated pricing game, pricing algorithms based on artificial intelligence (Q-learning) may consistently learn to charge supra-competitive prices. Although concerns on algorithmic collusion have arisen, little is known on underlying factors.
  arXiv  Detail & Related papers  (2021-02-18T03:28:41Z)
• Provably Efficient Algorithms for Multi-Objective Competitive RL [54.22598924633369]
  We study multi-objective reinforcement learning (RL) where an agent's reward is represented as a vector. In settings where an agent competes against opponents, its performance is measured by the distance of its average return vector to a target set. We develop statistically and computationally efficient algorithms to approach the associated target set.
  arXiv  Detail & Related papers  (2021-02-05T14:26:00Z)
• Provable Self-Play Algorithms for Competitive Reinforcement Learning [48.12602400021397]
  We study self-play in competitive reinforcement learning under the setting of Markov games. We show that a self-play algorithm achieves regret $tildemathcalO(sqrtT)$ after playing $T$ steps of the game. We also introduce an explore-then-exploit style algorithm, which achieves a slightly worse regret $tildemathcalO(T2/3)$, but is guaranteed to run in time even in the worst case.
  arXiv  Detail & Related papers  (2020-02-10T18:44:50Z)
• Adversarial Attacks on Linear Contextual Bandits [87.08004581867537]
  Malicious agents may have incentives to attack the bandit algorithm to induce it to perform a desired behavior. We show that a malicious agent can force a linear contextual bandit algorithm to pull any desired arm $T - o(T)$ times over a horizon of $T$ steps. We also investigate the case when a malicious agent is interested in affecting the behavior of the bandit algorithm in a single context.
  arXiv  Detail & Related papers  (2020-02-10T15:04:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.