HiBid: A Cross-Channel Constrained Bidding System with Budget Allocation by Hierarchical Offline Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2312.17503v2
• Date: Tue, 20 Aug 2024 08:09:26 GMT
• Title: HiBid: A Cross-Channel Constrained Bidding System with Budget Allocation by Hierarchical Offline Deep Reinforcement Learning
• Authors: Hao Wang, Bo Tang, Chi Harold Liu, Shangqin Mao, Jiahong Zhou, Zipeng Dai, Yaqi Sun, Qianlong Xie, Xingxing Wang, Dong Wang,
• Abstract summary: We propose a hierarchical offline deep reinforcement learning (DRL) framework called HiBid'' HiBid consists of a high-level planner equipped with auxiliary loss for non-competitive budget allocation. A CPC-guided action selection mechanism is introduced to satisfy the cross-channel CPC constraint.
• Score: 31.88174870851001
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Online display advertising platforms service numerous advertisers by providing real-time bidding (RTB) for the scale of billions of ad requests every day. The bidding strategy handles ad requests cross multiple channels to maximize the number of clicks under the set financial constraints, i.e., total budget and cost-per-click (CPC), etc. Different from existing works mainly focusing on single channel bidding, we explicitly consider cross-channel constrained bidding with budget allocation. Specifically, we propose a hierarchical offline deep reinforcement learning (DRL) framework called ``HiBid'', consisted of a high-level planner equipped with auxiliary loss for non-competitive budget allocation, and a data augmentation enhanced low-level executor for adaptive bidding strategy in response to allocated budgets. Additionally, a CPC-guided action selection mechanism is introduced to satisfy the cross-channel CPC constraint. Through extensive experiments on both the large-scale log data and online A/B testing, we confirm that HiBid outperforms six baselines in terms of the number of clicks, CPC satisfactory ratio, and return-on-investment (ROI). We also deploy HiBid on Meituan advertising platform to already service tens of thousands of advertisers every day.

Related papers

• RTBAgent: A LLM-based Agent System for Real-Time Bidding [11.49782135521099]
  Real-Time Bidding (RTB) enables advertisers to place competitive bids on impression opportunities instantaneously. To handle these challenges, RTBAgent is proposed as the first RTB agent system based on large language models (LLMs) We propose a two-step decision-making process and multi-memory retrieval mechanism, which enables RTBAgent to review historical decisions and transaction records.
  arXiv  Detail & Related papers  (2025-02-02T13:10:15Z)
• Offline Learning for Combinatorial Multi-armed Bandits [56.96242764723241]
  Off-CMAB is the first offline learning framework for CMAB. Off-CMAB combines pessimistic reward estimations with solvers. Experiments on synthetic and real-world datasets highlight the superior performance of CLCB.
  arXiv  Detail & Related papers  (2025-01-31T16:56:18Z)
• An Adaptable Budget Planner for Enhancing Budget-Constrained Auto-Bidding in Online Advertising [28.4314408199823]
  ABPlanner is a few-shot adaptable budget planner designed to improve budget-constrained auto-bidding. ABPlanner allocates the budget across all stages, allowing a low-level auto-bidder to bids based on the budget allocation plan. The adaptability of ABPlanner is achieved through a sequential decision-making approach, inspired by in-context reinforcement learning.
  arXiv  Detail & Related papers  (2025-01-26T08:00:23Z)
• Hierarchical Multi-agent Meta-Reinforcement Learning for Cross-channel Bidding [4.741091524027138]
  Real-time bidding (RTB) plays a pivotal role in online advertising ecosystems. Traditional approaches cannot effectively manage the dynamic budget allocation problem. We propose a hierarchical multi-agent reinforcement learning framework for multi-channel bidding optimization.
  arXiv  Detail & Related papers  (2024-12-26T05:26:30Z)
• ACQ: A Unified Framework for Automated Programmatic Creativity in Online Advertising [30.584160762498655]
  This paper proposes a two-stage framework named Automated Creatives Quota (ACQ) to achieve the automatic creation and deactivation of ad creatives. ACQ dynamically allocates the creative quota across multiple advertisers to maximize the revenue of the ad platform.
  arXiv  Detail & Related papers  (2024-12-09T03:00:57Z)
• Multi-Platform Budget Management in Ad Markets with Non-IC Auctions [6.037383467521294]
  In online advertising markets, budget-constrained advertisers acquire ad placements through repeated bidding in auctions on various platforms. We present a strategy for bidding optimally in a set of auctions that may or may not be incentive-compatible under the presence of budget constraints. Our strategy maximizes the expected total utility across auctions while satisfying the advertiser's budget constraints in expectation.
  arXiv  Detail & Related papers  (2023-06-12T18:21:10Z)
• VFed-SSD: Towards Practical Vertical Federated Advertising [53.08038962443853]
  We propose a semi-supervised split distillation framework VFed-SSD to alleviate the two limitations. Specifically, we develop a self-supervised task MatchedPair Detection (MPD) to exploit the vertically partitioned unlabeled data. Our framework provides an efficient federation-enhanced solution for real-time display advertising with minimal deploying cost and significant performance lift.
  arXiv  Detail & Related papers  (2022-05-31T17:45:30Z)
• A Cooperative-Competitive Multi-Agent Framework for Auto-bidding in Online Advertising [53.636153252400945]
  We propose a general Multi-Agent reinforcement learning framework for Auto-Bidding, namely MAAB, to learn the auto-bidding strategies. Our approach outperforms several baseline methods in terms of social welfare and guarantees the ad platform's revenue.
  arXiv  Detail & Related papers  (2021-06-11T08:07:14Z)
• Dynamic Knapsack Optimization Towards Efficient Multi-Channel Sequential Advertising [52.3825928886714]
  We formulate the sequential advertising strategy optimization as a dynamic knapsack problem. We propose a theoretically guaranteed bilevel optimization framework, which significantly reduces the solution space of the original optimization space. To improve the exploration efficiency of reinforcement learning, we also devise an effective action space reduction approach.
  arXiv  Detail & Related papers  (2020-06-29T18:50:35Z)
• Online Joint Bid/Daily Budget Optimization of Internet Advertising Campaigns [115.96295568115251]
  We study the problem of automating the online joint bid/daily budget optimization of pay-per-click advertising campaigns over multiple channels. For every campaign, we capture the dependency of the number of clicks on the bid and daily budget by Gaussian Processes. We design four algorithms and show that they suffer from a regret that is upper bounded with high probability as O(sqrtT) We present the results of the adoption of our algorithms in a real-world application with a daily average spent of 1,000 Euros for more than one year.
  arXiv  Detail & Related papers  (2020-03-03T11:07:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.