End-to-End User Behavior Retrieval in Click-Through RatePrediction Model

• URL: http://arxiv.org/abs/2108.04468v1
• Date: Tue, 10 Aug 2021 06:28:29 GMT
• Title: End-to-End User Behavior Retrieval in Click-Through RatePrediction Model
• Authors: Qiwei Chen, Changhua Pei, Shanshan Lv, Chao Li, Junfeng Ge, Wenwu Ou
• Abstract summary: We propose a locality-sensitive hashing (LSH) method called ETA which can greatly reduce the training and inference cost. We deploy ETA into a large-scale real world E-commerce system and achieve extra 3.1% improvements on GMV (Gross Merchandise Value) compared to a two-stage long user sequence CTR model.
• Score: 15.52581453176164
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Click-Through Rate (CTR) prediction is one of the core tasks in recommender systems (RS). It predicts a personalized click probability for each user-item pair. Recently, researchers have found that the performance of CTR model can be improved greatly by taking user behavior sequence into consideration, especially long-term user behavior sequence. The report on an e-commerce website shows that 23\% of users have more than 1000 clicks during the past 5 months. Though there are numerous works focus on modeling sequential user behaviors, few works can handle long-term user behavior sequence due to the strict inference time constraint in real world system. Two-stage methods are proposed to push the limit for better performance. At the first stage, an auxiliary task is designed to retrieve the top-$k$ similar items from long-term user behavior sequence. At the second stage, the classical attention mechanism is conducted between the candidate item and $k$ items selected in the first stage. However, information gap happens between retrieval stage and the main CTR task. This goal divergence can greatly diminishing the performance gain of long-term user sequence. In this paper, inspired by Reformer, we propose a locality-sensitive hashing (LSH) method called ETA (End-to-end Target Attention) which can greatly reduce the training and inference cost and make the end-to-end training with long-term user behavior sequence possible. Both offline and online experiments confirm the effectiveness of our model. We deploy ETA into a large-scale real world E-commerce system and achieve extra 3.1\% improvements on GMV (Gross Merchandise Value) compared to a two-stage long user sequence CTR model.

Related papers

• Multi-granularity Interest Retrieval and Refinement Network for Long-Term User Behavior Modeling in CTR Prediction [68.90783662117936]
  Click-through Rate (CTR) prediction is crucial for online personalization platforms. Recent advancements have shown that modeling rich user behaviors can significantly improve the performance of CTR prediction. We propose Multi-granularity Interest Retrieval and Refinement Network (MIRRN)
  arXiv  Detail & Related papers  (2024-11-22T15:29:05Z)
• TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou [28.809014888174932]
  We introduce TWIN-V2, an enhancement of SIM, where a divide-and-conquer approach is applied to compress life-cycle behaviors and uncover more accurate and diverse user interests. Under an efficient deployment framework, TWIN-V2 has been successfully deployed to the primary traffic that serves hundreds of millions of daily active users at Kuaishou.
  arXiv  Detail & Related papers  (2024-07-23T10:00:45Z)
• Deep Group Interest Modeling of Full Lifelong User Behaviors for CTR Prediction [32.35797628133921]
  Deep Group Interest Network (DGIN) is an end-to-end method to model the user's entire behavior history. DGIN grouping reduces the behavior length significantly, from O(104) to O(102) We identify a subset of behaviors that share the same item_id with the candidate item from the lifelong behavior sequence.
  arXiv  Detail & Related papers  (2023-11-15T06:36:11Z)
• Efficient Long Sequential User Data Modeling for Click-Through Rate Prediction [9.455385139394494]
  We propose an end-to-end paradigm to model long behavior sequences, which is able to achieve superior performance and remarkable cost-efficiency. Our contribution is three-fold: First, we propose a hashing-based efficient target attention (TA) network named ETA-Net to enable end-to-end user behavior retrieval. ETA-Net has been deployed on the recommender system of Taobao, and brought 1.8% lift on CTR and 3.1% lift on Gross Merchandise Value (GMV) compared to the SOTA two-stage methods.
  arXiv  Detail & Related papers  (2022-09-25T12:59:57Z)
• Recommender Transformers with Behavior Pathways [50.842316273120744]
  We build the Recommender Transformer (RETR) with a novel Pathway Attention mechanism. We empirically verify the effectiveness of RETR on seven real-world datasets.
  arXiv  Detail & Related papers  (2022-06-13T08:58:37Z)
• Sampling Is All You Need on Modeling Long-Term User Behaviors for CTR Prediction [15.97120392599086]
  We propose textbfM (textbfSampling-based textbfDeep textbfModeling), a simple yet effective sampling-based end-to-end approach for modeling long-term user behaviors. We show theoretically and experimentally that the proposed method performs on par with standard attention-based models on modeling long-term user behaviors.
  arXiv  Detail & Related papers  (2022-05-20T15:20:52Z)
• PinnerFormer: Sequence Modeling for User Representation at Pinterest [60.335384724891746]
  We introduce PinnerFormer, a user representation trained to predict a user's future long-term engagement. Unlike prior approaches, we adapt our modeling to a batch infrastructure via our new dense all-action loss. We show that by doing so, we significantly close the gap between batch user embeddings that are generated once a day and realtime user embeddings generated whenever a user takes an action.
  arXiv  Detail & Related papers  (2022-05-09T18:26:51Z)
• Sequential Search with Off-Policy Reinforcement Learning [48.88165680363482]
  We propose a highly scalable hybrid learning model that consists of an RNN learning framework and an attention model. As a novel optimization step, we fit multiple short user sequences in a single RNN pass within a training batch, by solving a greedy knapsack problem on the fly. We also explore the use of off-policy reinforcement learning in multi-session personalized search ranking.
  arXiv  Detail & Related papers  (2022-02-01T06:52:40Z)
• Dynamic Memory based Attention Network for Sequential Recommendation [79.5901228623551]
  We propose a novel long sequential recommendation model called Dynamic Memory-based Attention Network (DMAN) It segments the overall long behavior sequence into a series of sub-sequences, then trains the model and maintains a set of memory blocks to preserve long-term interests of users. Based on the dynamic memory, the user's short-term and long-term interests can be explicitly extracted and combined for efficient joint recommendation.
  arXiv  Detail & Related papers  (2021-02-18T11:08:54Z)
• Multi-Interactive Attention Network for Fine-grained Feature Learning in CTR Prediction [48.267995749975476]
  In the Click-Through Rate (CTR) prediction scenario, user's sequential behaviors are well utilized to capture the user interest. Existing methods mostly utilize attention on the behavior of users, which is not always suitable for CTR prediction. We propose a Multi-Interactive Attention Network (MIAN) to comprehensively extract the latent relationship among all kinds of fine-grained features.
  arXiv  Detail & Related papers  (2020-12-13T05:46:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.