Related papers: Learning to Allocate Incentives for Incentivized Advertising via Offline Model-Based Reinforcement Learning

Learning to Allocate Incentives for Incentivized Advertising via Offline Model-Based Reinforcement Learning

URL: http://arxiv.org/abs/2608.28065v1
Date: Fri, 28 Aug 2026 08:35:41 GMT
Title: Learning to Allocate Incentives for Incentivized Advertising via Offline Model-Based Reinforcement Learning
Abstract summary: In incentivized advertising, a platform promises users a bonus before observing downstream ad revenue, encouraging them to click and complete ads.<n>It must balance the incentive promised in advance against the revenue realized afterward.<n>We develop an offline model-based RL framework for cost-controllable sequential incentive allocation.
Score: 12.901979770261278
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Complete your ad view and grab a 5-cent bonus! In incentivized advertising, a platform promises users a bonus before observing downstream ad revenue, encouraging them to click and complete ads. It must balance the incentive promised in advance against the revenue realized afterward: insufficient incentives forfeit monetization opportunities, whereas excessive incentives reduce net profit. Because current incentives may also shape user expectations and future engagement, incentive allocation is a sequential decision problem with delayed revenue, cost sensitivity, and carryover effects. Existing work has not studied decision-making algorithms for this setting. Auto-bidding assumes available ad opportunities, while targeted promotion optimizes incentives outside the ad monetization pipeline. We formulate the problem as an MDP and develop an offline model-based RL framework for cost-controllable sequential incentive allocation. It learns a world model of user feedback and ad revenue, then performs conservative policy optimization. An independent counterfactual scorer evaluates each learned policy on held-out logs, enabling pre-launch selection without costly online exposure. Experiments on large-scale industrial data and online A/B tests show that the scorer provides a stable offline signal. The deployment path from causal inference to offline RL and then Offline-MBRL further validates the framework: MB-IQL improves per-user net profit by 7.96\% over TD3+BC, whereas reverting to plain IQL reduces it by 6.56\% (both $p<0.0001$).

Related papers

Optimizing Monetization Strategies for Generative AI Firms: Implications for Search Engagement [0.0]
We show that alternative advertising-supported monetization models can help GenAI firms recover costs while maintaining search query engagement.<n>We develop a framework wherein the introduction of advertising-supported monetization models influences user upgrading and downgrading decisions.<n>We show that affective and cognitive evaluations mediate preference for advertising-supported models, with temporal intrusiveness, but not visual, moderating these effects.
arXiv Detail & Related papers (2026-07-30T19:11:01Z)
Adaptive Ad Load Design for Sponsored Search Markets: Evidence, Theory, and Deployment [0.0]
Ad-load design is a central supply-side decision in sponsored search.<n>We study this trade-off using a large-scale randomized field experiment on an Android app store.<n>We deploy a novel adaptive algorithm -- exploration-augmented Locally Adaptive Ad Load (e-LAAL)
arXiv Detail & Related papers (2026-07-15T23:12:26Z)
TRACE: A Unified Rollout Budget Allocation Framework for Efficient Agentic Reinforcement Learning [52.67756371231985]
Reinforcement learning with verifiable rewards (RLVR) is a promising approach for enhancing reasoning and agentic behavior in large language models.<n>We introduce Tree Rollout Allocation for Contrastive Exploration (TRACE), a unified rollout allocation framework.<n>Technically, TRACE allocates rollout budget to both prompt roots and intermediate prefixes that are most likely to yield mixed terminal rewards.
arXiv Detail & Related papers (2026-06-09T17:16:03Z)
Multi-task Offline Reinforcement Learning for Online Advertising in Recommender Systems [54.709976343045824]
Current offline reinforcement learning (RL) methods face substantial challenges when applied to sparse advertising scenarios.<n>We propose MTORL, a novel multi-task offline RL model that targets two key objectives.<n>We employ multi-task learning to decode actions and rewards, simultaneously addressing channel recommendation and budget allocation.
arXiv Detail & Related papers (2025-06-29T05:05:13Z)
External Evaluation of Discrimination Mitigation Efforts in Meta's Ad Delivery [47.418845064441605]
We show that the Variance Reduction System (VRS) implemented by Meta does not meaningfully improve access to opportunities for individuals.<n>We then conduct experiments to evaluate VRS with real-world ads, and show that while VRS does reduce variance, it also raises advertiser costs.<n>We show our approach outperforms VRS by both increasing ad exposure for users from all groups and reducing cost to advertisers.
arXiv Detail & Related papers (2025-06-19T19:29:26Z)
Augmenting Unsupervised Reinforcement Learning with Self-Reference [63.68018737038331]
Humans possess the ability to draw on past experiences explicitly when learning new tasks. We propose the Self-Reference (SR) approach, an add-on module explicitly designed to leverage historical information. Our approach achieves state-of-the-art results in terms of Interquartile Mean (IQM) performance and Optimality Gap reduction on the Unsupervised Reinforcement Learning Benchmark.
arXiv Detail & Related papers (2023-11-16T09:07:34Z)
Adversarial Learning for Incentive Optimization in Mobile Payment Marketing [17.645000197183045]
Payment platforms hold large-scale marketing campaigns, which allocate incentives to encourage users to pay through their applications. To maximize the return on investment, incentive allocations are commonly solved in a two-stage procedure. We propose a bias correction adversarial network to overcome this obstacle.
arXiv Detail & Related papers (2021-12-28T07:54:39Z)
A framework for massive scale personalized promotion [18.12992386307048]
Technology companies building consumer-facing platforms may have access to massive-scale user population. Promotion with quantifiable incentive has become a popular approach for increasing active users on such platforms. This paper proposes a practical two-stage framework that can optimize the ROI of various massive-scale promotion campaigns.
arXiv Detail & Related papers (2021-08-27T03:03:18Z)
Offline Reinforcement Learning as Anti-Exploration [49.72457136766916]
We take inspiration from the literature on bonus-based exploration to design a new offline RL agent. The core idea is to subtract a prediction-based exploration bonus from the reward, instead of adding it for exploration. We show that our agent is competitive with the state of the art on a set of continuous control locomotion and manipulation tasks.
arXiv Detail & Related papers (2021-06-11T14:41:30Z)
Optimizing AD Pruning of Sponsored Search with Reinforcement Learning [14.583308909225552]
Industrial sponsored search system (SSS) can be logically divided into three modules: keywords matching, ad retrieving, and ranking. The problem we are going to address is: how to pick out the best $K$ items from $N$ candidates to maximize the system's revenue. We propose a novel model-free reinforcement learning approach to fixing this problem.
arXiv Detail & Related papers (2020-08-05T09:19:10Z)
Dynamic Knapsack Optimization Towards Efficient Multi-Channel Sequential Advertising [52.3825928886714]
We formulate the sequential advertising strategy optimization as a dynamic knapsack problem. We propose a theoretically guaranteed bilevel optimization framework, which significantly reduces the solution space of the original optimization space. To improve the exploration efficiency of reinforcement learning, we also devise an effective action space reduction approach.
arXiv Detail & Related papers (2020-06-29T18:50:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.