DRIVE: Distributional and Retrieval-Augmented Bidding with Value Evaluation

• URL: http://arxiv.org/abs/2606.14192v1
• Date: Fri, 12 Jun 2026 07:21:07 GMT
• Title: DRIVE: Distributional and Retrieval-Augmented Bidding with Value Evaluation
• Abstract summary: offline reinforcement learning and Transformer-based sequence modeling have shown promise for learning bidding policies from logged data.<n>We propose DRIVE, a unified Transformer-based framework that decouples candidate action generation from decision making for offline auto-bidding.<n>Experiments on AuctionNet and additional offline reinforcement learning benchmarks demonstrate that DRIVE consistently improves bidding performance and generalizes well across multiple Transformer-based methods.
• Score: 39.181584470241354
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Auto-bidding is a core component of real-time advertising systems, where decisions must optimize long-term performance under budget and cost constraints, while online exploration is prohibitively risky. Offline reinforcement learning and, more recently, Transformer-based sequence modeling have shown promise for learning bidding policies from logged data, but their unimodal and purely parametric formulations often collapse multiple effective bidding strategies into suboptimal averaged actions and perform unreliably under sparse or long-tail traffic. To mitigate these limitations, we propose DRIVE (Distributional and Retrieval-Augmented Bidding with Value Evaluation), a unified Transformer-based framework that decouples candidate action generation from decision making for offline auto-bidding. DRIVE combines distributional action modeling, retrieval-augmented candidate generation from high-quality historical decisions, and value-based evaluation to select the most promising bid at inference time. Extensive experiments on AuctionNet and additional offline reinforcement learning benchmarks demonstrate that DRIVE consistently improves bidding performance and generalizes well across multiple Transformer-based methods.

Related papers

• HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising [4.325040293861279]
  HOBA is a hierarchical reinforcement learning framework that decouples strategic reasoning, model selection, and bid execution across three time scales.<n> Experiments on the AuctionNet benchmark and a large-scale A/B test demonstrate consistent improvements over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-06-17T15:00:34Z)
• OLIVIA: Online Learning via Inference-time Action Adaptation for Decision Making in LLM ReAct Agents [74.20327254615854]
  Large language model agents interleave reasoning, action selection, and observation to solve sequential decision-making tasks.<n>Existing inference-time adaptation methods for LLM agents mainly rely on prompting or retrieval.<n>We propose OLIVIA, an inference-time action adaptation framework for ReAct-style agents.
  arXiv  Detail & Related papers  (2026-05-11T19:28:20Z)
• JD-BP: A Joint-Decision Generative Framework for Auto-Bidding and Pricing [15.898784873581677]
  Auto-bidding services optimize real-time bidding strategies for advertisers under key performance indicator (KPI) constraints.<n> uncertainties such as model prediction errors and feedback latency can cause bidding strategies to deviate from ex-post optimality.<n>We propose JD-BP, a Joint generative Decision framework for Bidding and Pricing.
  arXiv  Detail & Related papers  (2026-04-07T13:11:12Z)
• Breaking Determinism: Stochastic Modeling for Reliable Off-Policy Evaluation in Ad Auctions [16.315158617837646]
  This work contributes the first practical and validated framework for reliable Off-Policy Evaluation (OPE) in deterministic auction environments.<n>We introduce the first principled framework for OPE in deterministic auctions by repurposing the bid landscape model to approximate the propensity score.<n>We validate our approach on the AuctionNet simulation benchmark and against 2-weeks online A/B test from a large-scale industrial platform.
  arXiv  Detail & Related papers  (2025-12-03T01:37:42Z)
• Autobidding Arena: unified evaluation of the classical and RL-based autobidding algorithms [71.47275796833235]
  We present a standardized and transparent evaluation protocol for comparing classical and reinforcement learning autobidding algorithms.<n>We utilize the most recent open-source environment developed in the industry, which accurately emulates the bidding process.
  arXiv  Detail & Related papers  (2025-10-22T08:27:56Z)
• Learning to Route LLMs from Bandit Feedback: One Policy, Many Trade-offs [69.2486294522259]
  BaRP is a Bandit Routing-feedback with Preferences approach that trains under the same partial-feedback restriction as deployment.<n> Framed as a contextual bandit over prompt features and a user preference vector, our method simulates an online feedback setting during training and adapts its routing decisions to each new prompt.
  arXiv  Detail & Related papers  (2025-10-08T18:24:59Z)
• Steerable Adversarial Scenario Generation through Test-Time Preference Alignment [58.37104890690234]
  Adversarial scenario generation is a cost-effective approach for safety assessment of autonomous driving systems.<n>We introduce a new framework named textbfSteerable textbfAdversarial scenario textbfGEnerator (SAGE)<n>SAGE enables fine-grained test-time control over the trade-off between adversariality and realism without any retraining.
  arXiv  Detail & Related papers  (2025-09-24T13:27:35Z)
• Generative Bid Shading in Real-Time Bidding Advertising [7.7746704524695485]
  This paper introduces Generative Bid Shading(GBS) as an end-to-end generative model.<n>It incorporates an autoregressive approach to generate ratios by capturing stepwise residual reward models.<n>It has been deployed on the Meit platform serving billions of bid requests daily.
  arXiv  Detail & Related papers  (2025-08-06T03:34:49Z)
• Can Prompt Difficulty be Online Predicted for Accelerating RL Finetuning of Reasoning Models? [65.18157595903124]
  This work investigates iterative approximate evaluation for arbitrary prompts.<n>It introduces Model Predictive Prompt Selection (MoPPS), a Bayesian risk-predictive framework.<n>MoPPS reliably predicts prompt difficulty and accelerates training with significantly reduced rollouts.
  arXiv  Detail & Related papers  (2025-07-07T03:20:52Z)
• BAT: Benchmark for Auto-bidding Task [67.56067222427946]
  We present an auction benchmark encompassing the two most prevalent auction formats.<n>We implement a series of robust baselines on a novel dataset.<n>This benchmark provides a user-friendly and intuitive framework for researchers and practitioners to develop and refine innovative autobidding algorithms.
  arXiv  Detail & Related papers  (2025-05-13T12:12:34Z)
• On the Role of Feedback in Test-Time Scaling of Agentic AI Workflows [71.92083784393418]
  Agentic AI (systems that autonomously plan and act) are becoming widespread, yet their task success rate on complex tasks remains low.<n>Inference-time alignment relies on three components: sampling, evaluation, and feedback.<n>We introduce Iterative Agent Decoding (IAD), a procedure that repeatedly inserts feedback extracted from different forms of critiques.
  arXiv  Detail & Related papers  (2025-04-02T17:40:47Z)
• Data-Driven Online Interactive Bidding Strategy for Demand Response [0.30586855806896046]
  Demand response (DR) provides the services of peak shaving, enhancing the efficiency of renewable energy utilization with a short response period, and low cost. Various categories of DR are established, e.g. automated DR, incentive DR, emergency DR, and demand bidding. This paper determines the bidding and purchasing strategy simultaneously employing the smart meter data and functions. The results prove that when facing diverse situations the proposed model can earn the optimal profit via off/online learning the bidding rules and robustly making the proper bid.
  arXiv  Detail & Related papers  (2022-02-09T02:44:20Z)
• Optimal Bidding Strategy without Exploration in Real-time Bidding [14.035270361462576]
  maximizing utility with a budget constraint is the primary goal for advertisers in real-time bidding (RTB) systems. Previous works ignore the losing auctions to alleviate the difficulty with censored states. We propose a novel practical framework using the maximum entropy principle to imitate the behavior of the true distribution observed in real-time traffic.
  arXiv  Detail & Related papers  (2020-03-31T20:43:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.