HiBid: A Cross-Channel Constrained Bidding System with Budget Allocation by Hierarchical Offline Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2312.17503v2
• Date: Tue, 20 Aug 2024 08:09:26 GMT
• Title: HiBid: A Cross-Channel Constrained Bidding System with Budget Allocation by Hierarchical Offline Deep Reinforcement Learning
• Authors: Hao Wang, Bo Tang, Chi Harold Liu, Shangqin Mao, Jiahong Zhou, Zipeng Dai, Yaqi Sun, Qianlong Xie, Xingxing Wang, Dong Wang,
• Abstract summary: We propose a hierarchical offline deep reinforcement learning (DRL) framework called HiBid'' HiBid consists of a high-level planner equipped with auxiliary loss for non-competitive budget allocation. A CPC-guided action selection mechanism is introduced to satisfy the cross-channel CPC constraint.
• Score: 31.88174870851001
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Online display advertising platforms service numerous advertisers by providing real-time bidding (RTB) for the scale of billions of ad requests every day. The bidding strategy handles ad requests cross multiple channels to maximize the number of clicks under the set financial constraints, i.e., total budget and cost-per-click (CPC), etc. Different from existing works mainly focusing on single channel bidding, we explicitly consider cross-channel constrained bidding with budget allocation. Specifically, we propose a hierarchical offline deep reinforcement learning (DRL) framework called ``HiBid'', consisted of a high-level planner equipped with auxiliary loss for non-competitive budget allocation, and a data augmentation enhanced low-level executor for adaptive bidding strategy in response to allocated budgets. Additionally, a CPC-guided action selection mechanism is introduced to satisfy the cross-channel CPC constraint. Through extensive experiments on both the large-scale log data and online A/B testing, we confirm that HiBid outperforms six baselines in terms of the number of clicks, CPC satisfactory ratio, and return-on-investment (ROI). We also deploy HiBid on Meituan advertising platform to already service tens of thousands of advertisers every day.

Related papers

• Coordinated Dynamic Bidding in Repeated Second-Price Auctions with Budgets [17.937079224726073]
  We study coordinated online bidding algorithms in repeated second-price auctions with budgets. We propose algorithms that guarantee every client a higher utility than the best she can get under independent bidding.
  arXiv  Detail & Related papers  (2023-06-13T11:55:04Z)
• Multi-Platform Budget Management in Ad Markets with Non-IC Auctions [6.037383467521294]
  In online advertising markets, budget-constrained advertisers acquire ad placements through repeated bidding in auctions on various platforms. We present a strategy for bidding optimally in a set of auctions that may or may not be incentive-compatible under the presence of budget constraints. Our strategy maximizes the expected total utility across auctions while satisfying the advertiser's budget constraints in expectation.
  arXiv  Detail & Related papers  (2023-06-12T18:21:10Z)
• VFed-SSD: Towards Practical Vertical Federated Advertising [53.08038962443853]
  We propose a semi-supervised split distillation framework VFed-SSD to alleviate the two limitations. Specifically, we develop a self-supervised task MatchedPair Detection (MPD) to exploit the vertically partitioned unlabeled data. Our framework provides an efficient federation-enhanced solution for real-time display advertising with minimal deploying cost and significant performance lift.
  arXiv  Detail & Related papers  (2022-05-31T17:45:30Z)
• Bidding Agent Design in the LinkedIn Ad Marketplace [16.815498720115443]
  We establish a general optimization framework for the design of automated bidding agent in online marketplaces. As a result, the framework allows, for instance, the joint optimization of a group of ads across multiple platforms each running its own auction format. We share practical learnings of the deployed bidding system in the LinkedIn ad marketplace based on this framework.
  arXiv  Detail & Related papers  (2022-02-25T03:01:57Z)
• A Cooperative-Competitive Multi-Agent Framework for Auto-bidding in Online Advertising [53.636153252400945]
  We propose a general Multi-Agent reinforcement learning framework for Auto-Bidding, namely MAAB, to learn the auto-bidding strategies. Our approach outperforms several baseline methods in terms of social welfare and guarantees the ad platform's revenue.
  arXiv  Detail & Related papers  (2021-06-11T08:07:14Z)
• Dynamic Knapsack Optimization Towards Efficient Multi-Channel Sequential Advertising [52.3825928886714]
  We formulate the sequential advertising strategy optimization as a dynamic knapsack problem. We propose a theoretically guaranteed bilevel optimization framework, which significantly reduces the solution space of the original optimization space. To improve the exploration efficiency of reinforcement learning, we also devise an effective action space reduction approach.
  arXiv  Detail & Related papers  (2020-06-29T18:50:35Z)
• Online Joint Bid/Daily Budget Optimization of Internet Advertising Campaigns [115.96295568115251]
  We study the problem of automating the online joint bid/daily budget optimization of pay-per-click advertising campaigns over multiple channels. For every campaign, we capture the dependency of the number of clicks on the bid and daily budget by Gaussian Processes. We design four algorithms and show that they suffer from a regret that is upper bounded with high probability as O(sqrtT) We present the results of the adoption of our algorithms in a real-world application with a daily average spent of 1,000 Euros for more than one year.
  arXiv  Detail & Related papers  (2020-03-03T11:07:38Z)
• MoTiAC: Multi-Objective Actor-Critics for Real-Time Bidding [47.555870679348416]
  We propose a Multi-ecTive Actor-Critics algorithm named MoTiAC for the problem of bidding optimization with various goals. Unlike previous RL models, the proposed MoTiAC can simultaneously fulfill multi-objective tasks in complicated bidding environments.
  arXiv  Detail & Related papers  (2020-02-18T07:16:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.