Approximating Auction Equilibria with Reinforcement Learning

• URL: http://arxiv.org/abs/2410.13960v1
• Date: Thu, 17 Oct 2024 18:34:57 GMT
• Title: Approximating Auction Equilibria with Reinforcement Learning
• Authors: Pranjal Rawat,
• Abstract summary: This paper introduces a self-play based reinforcement learning approach that employs advanced algorithms to approximate Bayes-Nash equilibria. Through self-play, these algorithms can learn robust and near-optimal bidding strategies in auctions with known equilibria.
• Score: 0.0
• License:
• Abstract: Traditional methods for computing equilibria in auctions become computationally intractable as auction complexity increases, particularly in multi-item and dynamic auctions. This paper introduces a self-play based reinforcement learning approach that employs advanced algorithms such as Proximal Policy Optimization and Neural Fictitious Self-Play to approximate Bayes-Nash equilibria. This framework allows for continuous action spaces, high-dimensional information states, and delayed payoffs. Through self-play, these algorithms can learn robust and near-optimal bidding strategies in auctions with known equilibria, including those with symmetric and asymmetric valuations, private and interdependent values, and multi-round auctions.

Related papers

• Procurement Auctions via Approximately Optimal Submodular Optimization [53.93943270902349]
  We study procurement auctions, where an auctioneer seeks to acquire services from strategic sellers with private costs. Our goal is to design computationally efficient auctions that maximize the difference between the quality of the acquired services and the total cost of the sellers.
  arXiv  Detail & Related papers  (2024-11-20T18:06:55Z)
• Deep Reinforcement Learning for Sequential Combinatorial Auctions [40.89021064082742]
  Revenue-optimal auction design is a challenging problem with significant theoretical and practical implications. We propose a new reinforcement learning framework tailored for sequential auctions that leverages first-order gradients. Our approach achieves significant improvement in revenue over both analytical baselines and standard reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2024-07-10T20:00:22Z)
• Sequential Manipulation Against Rank Aggregation: Theory and Algorithm [119.57122943187086]
  We leverage an online attack on the vulnerable data collection process. From the game-theoretic perspective, the confrontation scenario is formulated as a distributionally robust game. The proposed method manipulates the results of rank aggregation methods in a sequential manner.
  arXiv  Detail & Related papers  (2024-07-02T03:31:21Z)
• Understanding Iterative Combinatorial Auction Designs via Multi-Agent Reinforcement Learning [10.41350502488723]
  We investigate whether multi-agent reinforcement learning algorithms can be used to understand iterative auctions. We find that MARL can indeed benefit auction analysis, but that deploying it effectively is nontrivial. We illustrate the promise of our resulting approach by using it to evaluate a specific rule change to a clock auction.
  arXiv  Detail & Related papers  (2024-02-29T18:16:13Z)
• Learning in Repeated Multi-Unit Pay-As-Bid Auctions [3.6294895527930504]
  We study the problem of bidding strategies in pay-as-bid (PAB) auctions from the perspective of single bidder. We show that a utility trick enables a time algorithm to solve the offline problem where competing bids are known in advance. We also present additional findings on the characterization of PAB equilibria.
  arXiv  Detail & Related papers  (2023-07-27T20:49:28Z)
• Autobidders with Budget and ROI Constraints: Efficiency, Regret, and Pacing Dynamics [53.62091043347035]
  We study a game between autobidding algorithms that compete in an online advertising platform. We propose a gradient-based learning algorithm that is guaranteed to satisfy all constraints and achieves vanishing individual regret.
  arXiv  Detail & Related papers  (2023-01-30T21:59:30Z)
• A unified stochastic approximation framework for learning in games [82.74514886461257]
  We develop a flexible approximation framework for analyzing the long-run behavior of learning in games (both continuous and finite) The proposed analysis template incorporates a wide array of popular learning algorithms, including gradient-based methods, exponential/multiplicative weights for learning in finite games, optimistic and bandit variants of the above, etc.
  arXiv  Detail & Related papers  (2022-06-08T14:30:38Z)
• A Context-Integrated Transformer-Based Neural Network for Auction Design [25.763612577196124]
  One of the central problems in auction design is developing an incentive-compatible mechanism that maximizes the auctioneer's expected revenue. We propose $mathttCITransNet$, a context-integrated transformer-based neural network for optimal auction design. We show by extensive experiments that $mathttCITransNet$ can recover the known optimal solutions in single-item settings, outperform strong baselines in multi-item auctions, and generalize well to cases other than those in training.
  arXiv  Detail & Related papers  (2022-01-29T03:47:00Z)
• Anti-Concentrated Confidence Bonuses for Scalable Exploration [57.91943847134011]
  Intrinsic rewards play a central role in handling the exploration-exploitation trade-off. We introduce emphanti-concentrated confidence bounds for efficiently approximating the elliptical bonus. We develop a practical variant for deep reinforcement learning that is competitive with contemporary intrinsic rewards on Atari benchmarks.
  arXiv  Detail & Related papers  (2021-10-21T15:25:15Z)
• ProportionNet: Balancing Fairness and Revenue for Auction Design with Deep Learning [55.76903822619047]
  We study the design of revenue-maximizing auctions with strong incentive guarantees. We extend techniques for approximating auctions using deep learning to address concerns of fairness while maintaining high revenue and strong incentive guarantees.
  arXiv  Detail & Related papers  (2020-10-13T13:54:21Z)
• Bid Prediction in Repeated Auctions with Learning [30.07778295477907]
  We consider the problem of bid prediction in repeated auctions using a dataset from a mainstream sponsored search auction marketplace. We propose the use of no-regret based econometrics for bid prediction, modeling players as no-regret learners with respect to a utility function. We show that the no-regret econometric methods perform comparable to state-of-the-art time-series machine learning methods.
  arXiv  Detail & Related papers  (2020-07-26T18:14:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.