Optimizing Online Advertising with Multi-Armed Bandits: Mitigating the Cold Start Problem under Auction Dynamics

• URL: http://arxiv.org/abs/2502.01867v1
• Date: Mon, 03 Feb 2025 22:33:24 GMT
• Title: Optimizing Online Advertising with Multi-Armed Bandits: Mitigating the Cold Start Problem under Auction Dynamics
• Authors: Anastasiia Soboleva, Andrey Pudovikov, Roman Snetkov, Alina Babenko, Egor Samosvat, Yuriy Dorn,
• Abstract summary: Insufficient behavioral data (clicks) makes accurate click-through rate forecasting of new ads challenging.<n>We develop a UCB-like algorithm under multi-armed bandit (MAB) setting for positional-based model.<n>In addition to increasing the platform's long-term profitability, we also propose a mechanism for maintaining short-term profits.
• Score: 0.1759252234439348
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Online advertising platforms often face a common challenge: the cold start problem. Insufficient behavioral data (clicks) makes accurate click-through rate (CTR) forecasting of new ads challenging. CTR for "old" items can also be significantly underestimated due to their early performance influencing their long-term behavior on the platform. The cold start problem has far-reaching implications for businesses, including missed long-term revenue opportunities. To mitigate this issue, we developed a UCB-like algorithm under multi-armed bandit (MAB) setting for positional-based model (PBM), specifically tailored to auction pay-per-click systems. Our proposed algorithm successfully combines theory and practice: we obtain theoretical upper estimates of budget regret, and conduct a series of experiments on synthetic and real-world data that confirm the applicability of the method on the real platform. In addition to increasing the platform's long-term profitability, we also propose a mechanism for maintaining short-term profits through controlled exploration and exploitation of items.

Related papers

• Nash Equilibrium Constrained Auto-bidding With Bi-level Reinforcement Learning [64.2367385090879]
  We propose a new formulation of the auto-bidding problem from the platform's perspective. It aims to maximize the social welfare of all advertisers under the $epsilon$-NE constraint. The NCB problem presents significant challenges due to its constrained bi-level structure and the typically large number of advertisers involved.
  arXiv  Detail & Related papers  (2025-03-13T12:25:36Z)
• Self-Regulation and Requesting Interventions [63.5863047447313]
  We propose an offline framework that trains a "helper" policy to request interventions. We score optimal intervention timing with PRMs and train the helper model on these labeled trajectories. This offline approach significantly reduces costly intervention calls during training.
  arXiv  Detail & Related papers  (2025-02-07T00:06:17Z)
• Online Learning in Contextual Second-Price Pay-Per-Click Auctions [47.06746975822902]
  We study online learning in contextual pay-per-click auctions where at each of the $T$ rounds, the learner receives some context along with a set of ads. The learner's goal is to minimize her regret, defined as the gap between her total revenue and that of an oracle strategy.
  arXiv  Detail & Related papers  (2023-10-08T07:04:22Z)
• Online Ad Procurement in Non-stationary Autobidding Worlds [10.871587311621974]
  We introduce a primal-dual algorithm for online decision making with multi-dimension decision variables, bandit feedback and long-term uncertain constraints. We show that our algorithm achieves low regret in many worlds when procurement outcomes are generated through procedures that are adversarial, adversarially corrupted, periodic, and ergodic.
  arXiv  Detail & Related papers  (2023-07-10T00:41:08Z)
• Online Learning under Budget and ROI Constraints via Weak Adaptivity [57.097119428915796]
  Existing primal-dual algorithms for constrained online learning problems rely on two fundamental assumptions. We show how such assumptions can be circumvented by endowing standard primal-dual templates with weakly adaptive regret minimizers. We prove the first best-of-both-worlds no-regret guarantees which hold in absence of the two aforementioned assumptions.
  arXiv  Detail & Related papers  (2023-02-02T16:30:33Z)
• Online Bidding Algorithms for Return-on-Spend Constrained Advertisers [10.500109788348732]
  This work explores efficient online algorithms for a single value-maximizing advertiser under an increasingly popular constraint: Return-on-Spend (RoS) We contribute a simple online algorithm that achieves near-optimal regret in expectation while always respecting the specified RoS constraint.
  arXiv  Detail & Related papers  (2022-08-29T16:49:24Z)
• Approaching sales forecasting using recurrent neural networks and transformers [57.43518732385863]
  We develop three alternatives to tackle the problem of forecasting the customer sales at day/store/item level using deep learning techniques. Our empirical results show how good performance can be achieved by using a simple sequence to sequence architecture with minimal data preprocessing effort. The proposed solution achieves a RMSLE of around 0.54, which is competitive with other more specific solutions to the problem proposed in the Kaggle competition.
  arXiv  Detail & Related papers  (2022-04-16T12:03:52Z)
• Mitigating Divergence of Latent Factors via Dual Ascent for Low Latency Event Prediction Models [0.739706777911384]
  Real-world content recommendation marketplaces exhibit certain behaviors and are imposed by constraints that are not always apparent in common static offline data sets. We present a systematic method to prevent model parameters from diverging by imposing a carefully chosen set of constraints on the model's latent vectors. We conduct an online experiment which shows a substantial reduction in the number of diverging instances, and a significant improvement to both user experience and revenue.
  arXiv  Detail & Related papers  (2021-11-15T16:09:48Z)
• Exploration in Online Advertising Systems with Deep Uncertainty-Aware Learning [26.24464382500032]
  We propose a novel Deep Uncertainty-Aware Learning (DUAL) method to learn click-through rate (CTR) prediction models. DUAL can be easily implemented on existing models and deployed in real-time systems with minimal extra computational overhead. We also present strategies to boost up long-term utilities such as the social welfare in advertising systems.
  arXiv  Detail & Related papers  (2020-11-25T17:23:52Z)
• Learning to Infer User Hidden States for Online Sequential Advertising [52.169666997331724]
  We propose our Deep Intents Sequential Advertising (DISA) method to address these issues. The key part of interpretability is to understand a consumer's purchase intent which is, however, unobservable (called hidden states)
  arXiv  Detail & Related papers  (2020-09-03T05:12:26Z)
• Dynamic Knapsack Optimization Towards Efficient Multi-Channel Sequential Advertising [52.3825928886714]
  We formulate the sequential advertising strategy optimization as a dynamic knapsack problem. We propose a theoretically guaranteed bilevel optimization framework, which significantly reduces the solution space of the original optimization space. To improve the exploration efficiency of reinforcement learning, we also devise an effective action space reduction approach.
  arXiv  Detail & Related papers  (2020-06-29T18:50:35Z)
• Optimal Bidding Strategy without Exploration in Real-time Bidding [14.035270361462576]
  maximizing utility with a budget constraint is the primary goal for advertisers in real-time bidding (RTB) systems. Previous works ignore the losing auctions to alleviate the difficulty with censored states. We propose a novel practical framework using the maximum entropy principle to imitate the behavior of the true distribution observed in real-time traffic.
  arXiv  Detail & Related papers  (2020-03-31T20:43:28Z)
• Adversarial Attacks on Linear Contextual Bandits [87.08004581867537]
  Malicious agents may have incentives to attack the bandit algorithm to induce it to perform a desired behavior. We show that a malicious agent can force a linear contextual bandit algorithm to pull any desired arm $T - o(T)$ times over a horizon of $T$ steps. We also investigate the case when a malicious agent is interested in affecting the behavior of the bandit algorithm in a single context.
  arXiv  Detail & Related papers  (2020-02-10T15:04:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.