A3M: Adaptive, Adversarial and Multi-Objective Learning for Strategic Bidding in Repeated Auctions

• URL: http://arxiv.org/abs/2606.28943v1
• Date: Sat, 27 Jun 2026 14:30:43 GMT
• Title: A3M: Adaptive, Adversarial and Multi-Objective Learning for Strategic Bidding in Repeated Auctions
• Abstract summary: We introduce the A3M framework, which integrates adaptive deep reinforcement learning, explicit adversarial reasoning, and principled multi-objective reward design for online auction strategy optimization.<n>A3M employs an actor-critic DRL backbone to dynamically balance exploration and exploitation, an opponent model for fictitious play against non-stationary adversaries, and a composite reward function to jointly maximize utility, auctioneer revenue, and fairness.<n>Results show that A3M reduces final regret by 30--40% in standard settings, maintains robust performance against adversarial strategy shifts, scales favorably with the number of units $K$, and enables tunable multi-objective trade
• Score: 19.323612677924732
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learning to bid in repeated multi-unit auctions with bandit feedback poses a fundamental challenge. Existing methods often rely on rigid explore-then-exploit schedules, assume stationary adversaries, and optimize solely for bidder utility, thereby limiting adaptability and strategic robustness. To address these limitations, we introduce the A3M framework, which integrates adaptive deep reinforcement learning (DRL), explicit adversarial reasoning, and principled multi-objective reward design for online auction strategy optimization. A3M employs an actor-critic DRL backbone to dynamically balance exploration and exploitation, an opponent model for fictitious play against non-stationary adversaries, and a composite reward function to jointly maximize utility, auctioneer revenue, and fairness. We provide the first comprehensive empirical evaluation of this integrated approach against established baselines in both discriminatory and uniform price auctions. Results show that A3M reduces final regret by 30--40\% in standard settings, maintains robust performance against adversarial strategy shifts, scales favorably with the number of units $K$, and enables tunable multi-objective trade-offs. An extensive ablation study confirms the necessity of each core component. Our work establishes A3M as a powerful and flexible framework for learning in complex auction environments.

Related papers

• GD$^2$PO: Mitigating Multi-Reward Conflicts via Group-Dynamic reward-Decoupled Policy Optimization [54.596224644751565]
  Post-training reinforcement learning (RL) increasingly relies on multi-dimensional rewards to cultivate comprehensive capabilities.<n>To address this, existing methods such as Group reward-Decoupled Policy Optimization (GDPO) decompose the overall score into independent reward groups, then compute the RL loss separately within each group.<n>We propose Group-Dynamic reward-Decoupled Policy Optimization (GD$2$PO), which employs a conflict-aware filtering mechanism to mask out rollouts suffering from severe reward-wise disagreement.
  arXiv  Detail & Related papers  (2026-06-15T14:19:28Z)
• AIPO: Learning to Reason from Active Interaction [54.10819421625103]
  AIPO enables the policy model to proactively consult three functional collaborative agents when encountering reasoning bottlenecks.<n>AIPO consistently improves reasoning performance, generalizes robustly across different policy models and RLVR algorithms, and effectively expands the reasoning capability boundary of the policy model.
  arXiv  Detail & Related papers  (2026-05-08T19:06:55Z)
• Training Language Models for Bilateral Trade with Private Information [47.25254982913256]
  Bilateral trade demands individual, strategic surplus rationality, and cooperation to realize gains from trade.<n>We develop a structured bargaining environment where large language models negotiate via tool calls within an event-driven simulator.<n>The environment serves two purposes: a benchmark for frontier models and as a training environment for open-weight models via reinforcement learning.
  arXiv  Detail & Related papers  (2026-04-10T03:04:20Z)
• Decomposable Reward Modeling and Realistic Environment Design for Reinforcement Learning-Based Forex Trading [0.0]
  This paper presents a modular reinforcement learning framework for Forex trading.<n>It includes a friction-aware execution engine that enforces strict anti-lookahead semantics, with observations at time t, execution at time t+1, and mark-to-market at time t+1.<n>It also incorporates realistic costs such as spread, commission, slippage, rollover financing, and margin-triggered liquidation.
  arXiv  Detail & Related papers  (2026-03-20T13:38:19Z)
• Mixture of Ranks with Degradation-Aware Routing for One-Step Real-World Image Super-Resolution [76.66229730098759]
  In real-world image super-resolution (Real-ISR), existing approaches mainly rely on fine-tuning pre-trained diffusion models.<n>We propose a Mixture-of-Ranks (MoR) architecture for single-step image super-resolution.<n>We introduce a fine-grained expert partitioning strategy that treats each rank in LoRA as an independent expert.
  arXiv  Detail & Related papers  (2025-11-20T04:11:44Z)
• The Bidding Games: Reinforcement Learning for MEV Extraction on Polygon Blockchain [0.11880231424287215]
  We present a reinforcement learning framework for MEV extraction on Polygon Atlas.<n>Our work establishes that reinforcement learning provides a critical advantage in high-frequency MEV environments.
  arXiv  Detail & Related papers  (2025-10-16T12:54:53Z)
• FURINA: Free from Unmergeable Router via LINear Aggregation of mixed experts [17.056585698418587]
  Mixture of Experts (MoE) has been successfully integrated into Low-Rank Adaptation (LoRA) for parameter-efficient fine-tuning.<n>A key limitation of existing MoE-LoRA methods is their reliance on a discrete router.<n>We propose FURINA, a novel Free from Unmergeable Router framework based on the LINear Aggregation of experts.
  arXiv  Detail & Related papers  (2025-09-18T12:22:32Z)
• Softmax with Regularization: Better Value Estimation in Multi-Agent Reinforcement Learning [72.28520951105207]
  Overestimation in $Q$-learning is an important problem that has been extensively studied in single-agent reinforcement learning. We propose a novel regularization-based update scheme that penalizes large joint action-values deviating from a baseline. We show that our method provides a consistent performance improvement on a set of challenging StarCraft II micromanagement tasks.
  arXiv  Detail & Related papers  (2021-03-22T14:18:39Z)
• MoTiAC: Multi-Objective Actor-Critics for Real-Time Bidding [47.555870679348416]
  We propose a Multi-ecTive Actor-Critics algorithm named MoTiAC for the problem of bidding optimization with various goals. Unlike previous RL models, the proposed MoTiAC can simultaneously fulfill multi-objective tasks in complicated bidding environments.
  arXiv  Detail & Related papers  (2020-02-18T07:16:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.