TBIN: Modeling Long Textual Behavior Data for CTR Prediction

• URL: http://arxiv.org/abs/2308.08483v1
• Date: Wed, 9 Aug 2023 03:48:41 GMT
• Title: TBIN: Modeling Long Textual Behavior Data for CTR Prediction
• Authors: Shuwei Chen, Xiang Li, Jian Dong, Jin Zhang, Yongkang Wang and Xingxing Wang
• Abstract summary: Click-through rate (CTR) prediction plays a pivotal role in the success of recommendations. Inspired by the recent thriving of language models (LMs), a surge of works improve prediction by organizing user behavior data in a textbftextual format. While promising, these works have to truncate the textual data to reduce the quadratic computational overhead of self-attention in LMs. In this paper, we propose a textbfTextual textbfBehavior-based textbfInterest Chunking textbfN
• Score: 15.056265935931377
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Click-through rate (CTR) prediction plays a pivotal role in the success of recommendations. Inspired by the recent thriving of language models (LMs), a surge of works improve prediction by organizing user behavior data in a \textbf{textual} format and using LMs to understand user interest at a semantic level. While promising, these works have to truncate the textual data to reduce the quadratic computational overhead of self-attention in LMs. However, it has been studied that long user behavior data can significantly benefit CTR prediction. In addition, these works typically condense user diverse interests into a single feature vector, which hinders the expressive capability of the model. In this paper, we propose a \textbf{T}extual \textbf{B}ehavior-based \textbf{I}nterest Chunking \textbf{N}etwork (TBIN), which tackles the above limitations by combining an efficient locality-sensitive hashing algorithm and a shifted chunk-based self-attention. The resulting user diverse interests are dynamically activated, producing user interest representation towards the target item. Finally, the results of both offline and online experiments on real-world food recommendation platform demonstrate the effectiveness of TBIN.

Related papers

• CTR-Sink: Attention Sink for Language Models in Click-Through Rate Prediction [42.92011330807996]
  $textitCTR-Sink$ is a novel framework introducing behavior-level attention sinks tailored for recommendation scenarios.<n>Inspired by attention sink theory, it constructs attention focus sinks and dynamically regulates attention aggregation via external information.
  arXiv  Detail & Related papers  (2025-08-05T17:30:34Z)
• CoKe: Customizable Fine-Grained Story Evaluation via Chain-of-Keyword Rationalization [90.15027447565427]
  Chain of thought (CoT) generates free-text explanations that help guide a model's predictions. Self-Consistency (SC) marginalizes predictions over multiple generated explanations. We propose $textbfC$hain-$textbfo$f-$textbfKe$ywords (CoKe)
  arXiv  Detail & Related papers  (2025-03-21T13:37:46Z)
• Multi-granularity Interest Retrieval and Refinement Network for Long-Term User Behavior Modeling in CTR Prediction [68.90783662117936]
  Click-through Rate (CTR) prediction is crucial for online personalization platforms. Recent advancements have shown that modeling rich user behaviors can significantly improve the performance of CTR prediction. We propose Multi-granularity Interest Retrieval and Refinement Network (MIRRN)
  arXiv  Detail & Related papers  (2024-11-22T15:29:05Z)
• LLM-assisted Explicit and Implicit Multi-interest Learning Framework for Sequential Recommendation [50.98046887582194]
  We propose an explicit and implicit multi-interest learning framework to model user interests on two levels: behavior and semantics. The proposed EIMF framework effectively and efficiently combines small models with LLM to improve the accuracy of multi-interest modeling.
  arXiv  Detail & Related papers  (2024-11-14T13:00:23Z)
• ELCoRec: Enhance Language Understanding with Co-Propagation of Numerical and Categorical Features for Recommendation [38.64175351885443]
  Large language models have been flourishing in the natural language processing (NLP) domain. Despite the intelligence shown by the recommendation-oriented finetuned models, LLMs struggle to fully understand the user behavior patterns. Existing works only fine-tune a sole LLM on given text data without introducing that important information to it.
  arXiv  Detail & Related papers  (2024-06-27T01:37:57Z)
• CELA: Cost-Efficient Language Model Alignment for CTR Prediction [71.85120354973073]
  Click-Through Rate (CTR) prediction holds a paramount position in recommender systems. Recent efforts have sought to mitigate these challenges by integrating Pre-trained Language Models (PLMs) We propose textbfCost-textbfEfficient textbfLanguage Model textbfAlignment (textbfCELA) for CTR prediction.
  arXiv  Detail & Related papers  (2024-05-17T07:43:25Z)
• Deep Group Interest Modeling of Full Lifelong User Behaviors for CTR Prediction [32.35797628133921]
  Deep Group Interest Network (DGIN) is an end-to-end method to model the user's entire behavior history. DGIN grouping reduces the behavior length significantly, from O(104) to O(102) We identify a subset of behaviors that share the same item_id with the candidate item from the lifelong behavior sequence.
  arXiv  Detail & Related papers  (2023-11-15T06:36:11Z)
• DELTA: Dynamic Embedding Learning with Truncated Conscious Attention for CTR Prediction [61.68415731896613]
  Click-Through Rate (CTR) prediction is a pivotal task in product and content recommendation. We propose a model that enables Dynamic Embedding Learning with Truncated Conscious Attention for CTR prediction.
  arXiv  Detail & Related papers  (2023-05-03T12:34:45Z)
• Meta-Wrapper: Differentiable Wrapping Operator for User Interest Selection in CTR Prediction [97.99938802797377]
  Click-through rate (CTR) prediction, whose goal is to predict the probability of the user to click on an item, has become increasingly significant in recommender systems. Recent deep learning models with the ability to automatically extract the user interest from his/her behaviors have achieved great success. We propose a novel approach under the framework of the wrapper method, which is named Meta-Wrapper.
  arXiv  Detail & Related papers  (2022-06-28T03:28:15Z)
• Sampling Is All You Need on Modeling Long-Term User Behaviors for CTR Prediction [15.97120392599086]
  We propose textbfM (textbfSampling-based textbfDeep textbfModeling), a simple yet effective sampling-based end-to-end approach for modeling long-term user behaviors. We show theoretically and experimentally that the proposed method performs on par with standard attention-based models on modeling long-term user behaviors.
  arXiv  Detail & Related papers  (2022-05-20T15:20:52Z)
• Masked Transformer for Neighhourhood-aware Click-Through Rate Prediction [74.52904110197004]
  We propose Neighbor-Interaction based CTR prediction, which put this task into a Heterogeneous Information Network (HIN) setting. In order to enhance the representation of the local neighbourhood, we consider four types of topological interaction among the nodes. We conduct comprehensive experiments on two real world datasets and the experimental results show that our proposed method outperforms state-of-the-art CTR models significantly.
  arXiv  Detail & Related papers  (2022-01-25T12:44:23Z)
• Sequential Recommendation via Stochastic Self-Attention [68.52192964559829]
  Transformer-based approaches embed items as vectors and use dot-product self-attention to measure the relationship between items. We propose a novel textbfSTOchastic textbfSelf-textbfAttention(STOSA) to overcome these issues. We devise a novel Wasserstein Self-Attention module to characterize item-item position-wise relationships in sequences.
  arXiv  Detail & Related papers  (2022-01-16T12:38:45Z)
• Multi-Interactive Attention Network for Fine-grained Feature Learning in CTR Prediction [48.267995749975476]
  In the Click-Through Rate (CTR) prediction scenario, user's sequential behaviors are well utilized to capture the user interest. Existing methods mostly utilize attention on the behavior of users, which is not always suitable for CTR prediction. We propose a Multi-Interactive Attention Network (MIAN) to comprehensively extract the latent relationship among all kinds of fine-grained features.
  arXiv  Detail & Related papers  (2020-12-13T05:46:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.