Asymptotically Unbiased Estimation for Delayed Feedback Modeling via Label Correction

• URL: http://arxiv.org/abs/2202.06472v2
• Date: Tue, 15 Feb 2022 04:42:22 GMT
• Title: Asymptotically Unbiased Estimation for Delayed Feedback Modeling via Label Correction
• Authors: Yu Chen, Jiaqi Jin, Hui Zhao, Pengjie Wang, Guojun Liu, Jian Xu and Bo Zheng
• Abstract summary: Delayed feedback is crucial for the conversion rate prediction in online advertising. Previous delayed feedback modeling methods balance the trade-off between waiting for accurate labels and consuming fresh feedback. We propose a new method, DElayed Feedback modeling with UnbiaSed Estimation, (DEFUSE), which aim to respectively correct the importance weights of the immediate positive, the fake negative, the real negative, and the delay positive samples.
• Score: 14.462884375151045
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Alleviating the delayed feedback problem is of crucial importance for the conversion rate(CVR) prediction in online advertising. Previous delayed feedback modeling methods using an observation window to balance the trade-off between waiting for accurate labels and consuming fresh feedback. Moreover, to estimate CVR upon the freshly observed but biased distribution with fake negatives, the importance sampling is widely used to reduce the distribution bias. While effective, we argue that previous approaches falsely treat fake negative samples as real negative during the importance weighting and have not fully utilized the observed positive samples, leading to suboptimal performance. In this work, we propose a new method, DElayed Feedback modeling with UnbiaSed Estimation, (DEFUSE), which aim to respectively correct the importance weights of the immediate positive, the fake negative, the real negative, and the delay positive samples at finer granularity. Specifically, we propose a two-step optimization approach that first infers the probability of fake negatives among observed negatives before applying importance sampling. To fully exploit the ground-truth immediate positives from the observed distribution, we further develop a bi-distribution modeling framework to jointly model the unbiased immediate positives and the biased delay conversions. Experimental results on both public and our industrial datasets validate the superiority of DEFUSE. Codes are available at https://github.com/ychen216/DEFUSE.git.

Related papers

• Contrastive Learning with Negative Sampling Correction [52.990001829393506]
  We propose a novel contrastive learning method named Positive-Unlabeled Contrastive Learning (PUCL) PUCL treats the generated negative samples as unlabeled samples and uses information from positive samples to correct bias in contrastive loss. PUCL can be applied to general contrastive learning problems and outperforms state-of-the-art methods on various image and graph classification tasks.
  arXiv  Detail & Related papers  (2024-01-13T11:18:18Z)
• Mitigating Pooling Bias in E-commerce Search via False Negative Estimation [25.40402675846542]
  Bias-mitigating Hard Negative Sampling is a novel negative sampling strategy tailored to identify and adjust for false negatives. Our experiments in the search setting confirm BHNS as effective for practical e-commerce use.
  arXiv  Detail & Related papers  (2023-11-11T00:22:57Z)
• DPR: An Algorithm Mitigate Bias Accumulation in Recommendation feedback loops [41.21024436158042]
  We study the negative impact of feedback loops and unknown exposure mechanisms on recommendation quality and user experience. We propose Dynamic Personalized Ranking (textbfDPR), an unbiased algorithm that uses dynamic re-weighting to mitigate the cross-effects. We show theoretically that our approach mitigates the negative effects of feedback loops and unknown exposure mechanisms.
  arXiv  Detail & Related papers  (2023-11-10T04:36:00Z)
• Your Negative May not Be True Negative: Boosting Image-Text Matching with False Negative Elimination [62.18768931714238]
  We propose a novel False Negative Elimination (FNE) strategy to select negatives via sampling. The results demonstrate the superiority of our proposed false negative elimination strategy.
  arXiv  Detail & Related papers  (2023-08-08T16:31:43Z)
• Generating Negative Samples for Sequential Recommendation [83.60655196391855]
  We propose to Generate Negative Samples (items) for Sequential Recommendation (SR) A negative item is sampled at each time step based on the current SR model's learned user preferences toward items. Experiments on four public datasets verify the importance of providing high-quality negative samples for SR.
  arXiv  Detail & Related papers  (2022-08-07T05:44:13Z)
• Rethinking InfoNCE: How Many Negative Samples Do You Need? [54.146208195806636]
  We study how many negative samples are optimal for InfoNCE in different scenarios via a semi-quantitative theoretical framework. We estimate the optimal negative sampling ratio using the $K$ value that maximizes the training effectiveness function.
  arXiv  Detail & Related papers  (2021-05-27T08:38:29Z)
• Real Negatives Matter: Continuous Training with Real Negatives for Delayed Feedback Modeling [10.828167195122072]
  We propose DElayed FEedback modeling with Real negatives (DEFER) method to address these issues. The ingestion of real negatives ensures the observed feature distribution is equivalent to the actual distribution, thus reducing the bias. DEFER have been deployed in the display advertising system of Alibaba, obtaining over 6.4% improvement on CVR in several scenarios.
  arXiv  Detail & Related papers  (2021-04-29T05:37:34Z)
• Capturing Delayed Feedback in Conversion Rate Prediction via Elapsed-Time Sampling [29.77426549280091]
  Conversion rate (CVR) prediction is one of the most critical tasks for digital display advertising. We propose Elapsed-Time Sampling Delayed Feedback Model (ES-DFM), which models the relationship between the observed conversion distribution and the true conversion distribution.
  arXiv  Detail & Related papers  (2020-12-06T12:20:50Z)
• Positive-Congruent Training: Towards Regression-Free Model Updates [87.25247195148187]
  In image classification, sample-wise inconsistencies appear as "negative flips" A new model incorrectly predicts the output for a test sample that was correctly classified by the old (reference) model. We propose a simple approach for PC training, Focal Distillation, which enforces congruence with the reference model.
  arXiv  Detail & Related papers  (2020-11-18T09:00:44Z)
• Understanding Negative Sampling in Graph Representation Learning [87.35038268508414]
  We show that negative sampling is as important as positive sampling in determining the optimization objective and the resulted variance. We propose Metropolis-Hastings (MCNS) to approximate the positive distribution with self-contrast approximation and accelerate negative sampling by Metropolis-Hastings. We evaluate our method on 5 datasets that cover extensive downstream graph learning tasks, including link prediction, node classification and personalized recommendation.
  arXiv  Detail & Related papers  (2020-05-20T06:25:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.