By Fair Means or Foul: Quantifying Collusion in a Market Simulation with Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2406.02650v1
• Date: Tue, 4 Jun 2024 15:35:08 GMT
• Title: By Fair Means or Foul: Quantifying Collusion in a Market Simulation with Deep Reinforcement Learning
• Authors: Michael Schlechtinger, Damaris Kosack, Franz Krause, Heiko Paulheim,
• Abstract summary: This research employs an experimental oligopoly model of repeated price competition. We investigate the strategies and emerging pricing patterns developed by the agents, which may lead to a collusive outcome. Our findings indicate that RL-based AI agents converge to a collusive state characterized by the charging of supracompetitive prices.
• Score: 1.5249435285717095
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the rapidly evolving landscape of eCommerce, Artificial Intelligence (AI) based pricing algorithms, particularly those utilizing Reinforcement Learning (RL), are becoming increasingly prevalent. This rise has led to an inextricable pricing situation with the potential for market collusion. Our research employs an experimental oligopoly model of repeated price competition, systematically varying the environment to cover scenarios from basic economic theory to subjective consumer demand preferences. We also introduce a novel demand framework that enables the implementation of various demand models, allowing for a weighted blending of different models. In contrast to existing research in this domain, we aim to investigate the strategies and emerging pricing patterns developed by the agents, which may lead to a collusive outcome. Furthermore, we investigate a scenario where agents cannot observe their competitors' prices. Finally, we provide a comprehensive legal analysis across all scenarios. Our findings indicate that RL-based AI agents converge to a collusive state characterized by the charging of supracompetitive prices, without necessarily requiring inter-agent communication. Implementing alternative RL algorithms, altering the number of agents or simulation settings, and restricting the scope of the agents' observation space does not significantly impact the collusive market outcome behavior.

Related papers

• Dual-Agent Deep Reinforcement Learning for Dynamic Pricing and Replenishment [15.273192037219077]
  We study the dynamic pricing and replenishment problems under inconsistent decision frequencies. We integrate a decision tree-based machine learning approach, trained on comprehensive market data. In this approach, two agents handle pricing and inventory and are updated on different scales.
  arXiv  Detail & Related papers  (2024-10-28T15:12:04Z)
• Deep Reinforcement Learning Agents for Strategic Production Policies in Microeconomic Market Simulations [1.6499388997661122]
  We propose a DRL-based approach to obtain an effective policy in competitive markets with multiple producers. Our framework enables agents to learn adaptive production policies to several simulations that consistently outperform static and random strategies. The results show that agents trained with DRL can strategically adjust production levels to maximize long-term profitability.
  arXiv  Detail & Related papers  (2024-10-27T18:38:05Z)
• Refined Mechanism Design for Approximately Structured Priors via Active Regression [50.71772232237571]
  We consider the problem of a revenue-maximizing seller with a large number of items for sale to $n$ strategic bidders. It is well-known that optimal and even approximately-optimal mechanisms for this setting are notoriously difficult to characterize or compute.
  arXiv  Detail & Related papers  (2023-10-11T20:34:17Z)
• "Guinea Pig Trials" Utilizing GPT: A Novel Smart Agent-Based Modeling Approach for Studying Firm Competition and Collusion [10.721432974840429]
  We propose an innovative framework called Smart Agent-Based Modeling (SABM), wherein smart agents represent firms and interact with one another. Smart agents possess an extensive knowledge base for decision-making and exhibit human-like strategic abilities, surpassing traditional ABM agents. Our results demonstrate that, in the absence of communication, smart agents consistently reach tacit collusion, leading to prices converging at levels higher than the Bertrand equilibrium price but lower than monopoly or cartel prices.
  arXiv  Detail & Related papers  (2023-08-21T18:42:17Z)
• Structured Dynamic Pricing: Optimal Regret in a Global Shrinkage Model [50.06663781566795]
  We consider a dynamic model with the consumers' preferences as well as price sensitivity varying over time. We measure the performance of a dynamic pricing policy via regret, which is the expected revenue loss compared to a clairvoyant that knows the sequence of model parameters in advance. Our regret analysis results not only demonstrate optimality of the proposed policy but also show that for policy planning it is essential to incorporate available structural information.
  arXiv  Detail & Related papers  (2023-03-28T00:23:23Z)
• Towards Multi-Agent Reinforcement Learning driven Over-The-Counter Market Simulations [16.48389671789281]
  We study a game between liquidity provider and liquidity taker agents interacting in an over-the-counter market. By playing against each other, our deep-reinforcement-learning-driven agents learn emergent behaviors. We show convergence rates for our multi-agent policy gradient algorithm under a transitivity assumption.
  arXiv  Detail & Related papers  (2022-10-13T17:06:08Z)
• Modeling Bounded Rationality in Multi-Agent Simulations Using Rationally Inattentive Reinforcement Learning [85.86440477005523]
  We study more human-like RL agents which incorporate an established model of human-irrationality, the Rational Inattention (RI) model. RIRL models the cost of cognitive information processing using mutual information. We show that using RIRL yields a rich spectrum of new equilibrium behaviors that differ from those found under rational assumptions.
  arXiv  Detail & Related papers  (2022-01-18T20:54:00Z)
• Finding General Equilibria in Many-Agent Economic Simulations Using Deep Reinforcement Learning [72.23843557783533]
  We show that deep reinforcement learning can discover stable solutions that are epsilon-Nash equilibria for a meta-game over agent types. Our approach is more flexible and does not need unrealistic assumptions, e.g., market clearing. We demonstrate our approach in real-business-cycle models, a representative family of DGE models, with 100 worker-consumers, 10 firms, and a government who taxes and redistributes.
  arXiv  Detail & Related papers  (2022-01-03T17:00:17Z)
• Explore and Control with Adversarial Surprise [78.41972292110967]
  Reinforcement learning (RL) provides a framework for learning goal-directed policies given user-specified rewards. We propose a new unsupervised RL technique based on an adversarial game which pits two policies against each other to compete over the amount of surprise an RL agent experiences. We show that our method leads to the emergence of complex skills by exhibiting clear phase transitions.
  arXiv  Detail & Related papers  (2021-07-12T17:58:40Z)
• VCG Mechanism Design with Unknown Agent Values under Stochastic Bandit Feedback [104.06766271716774]
  We study a multi-round welfare-maximising mechanism design problem in instances where agents do not know their values. We first define three notions of regret for the welfare, the individual utilities of each agent and that of the mechanism. Our framework also provides flexibility to control the pricing scheme so as to trade-off between the agent and seller regrets.
  arXiv  Detail & Related papers  (2020-04-19T18:00:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.