Capturing Delayed Feedback in Conversion Rate Prediction via Elapsed-Time Sampling

• URL: http://arxiv.org/abs/2012.03245v2
• Date: Thu, 21 Jan 2021 10:07:29 GMT
• Title: Capturing Delayed Feedback in Conversion Rate Prediction via Elapsed-Time Sampling
• Authors: Jia-Qi Yang, Xiang Li, Shuguang Han, Tao Zhuang, De-Chuan Zhan, Xiaoyi Zeng, Bin Tong
• Abstract summary: Conversion rate (CVR) prediction is one of the most critical tasks for digital display advertising. We propose Elapsed-Time Sampling Delayed Feedback Model (ES-DFM), which models the relationship between the observed conversion distribution and the true conversion distribution.
• Score: 29.77426549280091
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conversion rate (CVR) prediction is one of the most critical tasks for digital display advertising. Commercial systems often require to update models in an online learning manner to catch up with the evolving data distribution. However, conversions usually do not happen immediately after a user click. This may result in inaccurate labeling, which is called delayed feedback problem. In previous studies, delayed feedback problem is handled either by waiting positive label for a long period of time, or by consuming the negative sample on its arrival and then insert a positive duplicate when a conversion happens later. Indeed, there is a trade-off between waiting for more accurate labels and utilizing fresh data, which is not considered in existing works. To strike a balance in this trade-off, we propose Elapsed-Time Sampling Delayed Feedback Model (ES-DFM), which models the relationship between the observed conversion distribution and the true conversion distribution. Then we optimize the expectation of true conversion distribution via importance sampling under the elapsed-time sampling distribution. We further estimate the importance weight for each instance, which is used as the weight of loss function in CVR prediction. To demonstrate the effectiveness of ES-DFM, we conduct extensive experiments on a public data and a private industrial dataset. Experimental results confirm that our method consistently outperforms the previous state-of-the-art results.

Related papers

• DOTA: Distributional Test-Time Adaptation of Vision-Language Models [52.98590762456236]
  Training-free test-time dynamic adapter (TDA) is a promising approach to address this issue. We propose a simple yet effective method for DistributiOnal Test-time Adaptation (Dota) Dota continually estimates the distributions of test samples, allowing the model to continually adapt to the deployment environment.
  arXiv  Detail & Related papers  (2024-09-28T15:03:28Z)
• Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting [55.17761802332469]
  Test-time adaptation (TTA) seeks to tackle potential distribution shifts between training and test data by adapting a given model w.r.t. any test sample. Prior methods perform backpropagation for each test sample, resulting in unbearable optimization costs to many applications. We propose an Efficient Anti-Forgetting Test-Time Adaptation (EATA) method which develops an active sample selection criterion to identify reliable and non-redundant samples.
  arXiv  Detail & Related papers  (2024-03-18T05:49:45Z)
• Data Attribution for Diffusion Models: Timestep-induced Bias in Influence Estimation [53.27596811146316]
  Diffusion models operate over a sequence of timesteps instead of instantaneous input-output relationships in previous contexts. We present Diffusion-TracIn that incorporates this temporal dynamics and observe that samples' loss gradient norms are highly dependent on timestep. We introduce Diffusion-ReTrac as a re-normalized adaptation that enables the retrieval of training samples more targeted to the test sample of interest.
  arXiv  Detail & Related papers  (2024-01-17T07:58:18Z)
• Data Feedback Loops: Model-driven Amplification of Dataset Biases [9.773315369593876]
  We formalize a system where interactions with one model are recorded as history and scraped as training data in the future. We analyze its stability over time by tracking changes to a test-time bias statistic. We find that the degree of bias amplification is closely linked to whether the model's outputs behave like samples from the training distribution.
  arXiv  Detail & Related papers  (2022-09-08T17:35:51Z)
• Generalized Delayed Feedback Model with Post-Click Information in Recommender Systems [37.72697954740977]
  We show that post-click user behaviors are also informative to conversion rate prediction and can be used to improve timeliness. We propose a generalized delayed feedback model (GDFM) that unifies both post-click behaviors and early conversions as post-click information.
  arXiv  Detail & Related papers  (2022-06-01T11:17:01Z)
• Asymptotically Unbiased Estimation for Delayed Feedback Modeling via Label Correction [14.462884375151045]
  Delayed feedback is crucial for the conversion rate prediction in online advertising. Previous delayed feedback modeling methods balance the trade-off between waiting for accurate labels and consuming fresh feedback. We propose a new method, DElayed Feedback modeling with UnbiaSed Estimation, (DEFUSE), which aim to respectively correct the importance weights of the immediate positive, the fake negative, the real negative, and the delay positive samples.
  arXiv  Detail & Related papers  (2022-02-14T03:31:09Z)
• Predicting with Confidence on Unseen Distributions [90.68414180153897]
  We connect domain adaptation and predictive uncertainty literature to predict model accuracy on challenging unseen distributions. We find that the difference of confidences (DoC) of a classifier's predictions successfully estimates the classifier's performance change over a variety of shifts. We specifically investigate the distinction between synthetic and natural distribution shifts and observe that despite its simplicity DoC consistently outperforms other quantifications of distributional difference.
  arXiv  Detail & Related papers  (2021-07-07T15:50:18Z)
• Real Negatives Matter: Continuous Training with Real Negatives for Delayed Feedback Modeling [10.828167195122072]
  We propose DElayed FEedback modeling with Real negatives (DEFER) method to address these issues. The ingestion of real negatives ensures the observed feature distribution is equivalent to the actual distribution, thus reducing the bias. DEFER have been deployed in the display advertising system of Alibaba, obtaining over 6.4% improvement on CVR in several scenarios.
  arXiv  Detail & Related papers  (2021-04-29T05:37:34Z)
• Time-Series Imputation with Wasserstein Interpolation for Optimal Look-Ahead-Bias and Variance Tradeoff [66.59869239999459]
  In finance, imputation of missing returns may be applied prior to training a portfolio optimization model. There is an inherent trade-off between the look-ahead-bias of using the full data set for imputation and the larger variance in the imputation from using only the training data. We propose a Bayesian posterior consensus distribution which optimally controls the variance and look-ahead-bias trade-off in the imputation.
  arXiv  Detail & Related papers  (2021-02-25T09:05:35Z)
• Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift [81.74795324629712]
  We call prediction-time batch normalization, which significantly improves model accuracy and calibration under covariate shift. We show that prediction-time batch normalization provides complementary benefits to existing state-of-the-art approaches for improving robustness. The method has mixed results when used alongside pre-training, and does not seem to perform as well under more natural types of dataset shift.
  arXiv  Detail & Related papers  (2020-06-19T05:08:43Z)
• A Feedback Shift Correction in Predicting Conversion Rates under Delayed Feedback [6.38500614968955]
  In display advertising, predicting the conversion rate is fundamental to estimating the value of displaying the advertisement. There is a relatively long time delay between a click and its resultant conversion. Because of the delayed feedback, some positive instances at the training period are labeled as negative.
  arXiv  Detail & Related papers  (2020-02-06T02:05:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.