PRECISE: Pre-training Sequential Recommenders with Collaborative and Semantic Information

• URL: http://arxiv.org/abs/2412.06308v1
• Date: Mon, 09 Dec 2024 08:55:48 GMT
• Title: PRECISE: Pre-training Sequential Recommenders with Collaborative and Semantic Information
• Authors: Chonggang Song, Chunxu Shen, Hao Gu, Yaoming Wu, Lingling Yi, Jie Wen, Chuan Chen,
• Abstract summary: Real-world recommendation systems commonly offer diverse content scenarios for users to interact with.<n>It is infeasible to utilize a single unified recommendation model to meet the requirements of all scenarios.<n>We propose a novel pre-training framework for sequential recommendation, termed PRECISE.
• Score: 12.033363057768137
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Real-world recommendation systems commonly offer diverse content scenarios for users to interact with. Considering the enormous number of users in industrial platforms, it is infeasible to utilize a single unified recommendation model to meet the requirements of all scenarios. Usually, separate recommendation pipelines are established for each distinct scenario. This practice leads to challenges in comprehensively grasping users' interests. Recent research endeavors have been made to tackle this problem by pre-training models to encapsulate the overall interests of users. Traditional pre-trained recommendation models mainly capture user interests by leveraging collaborative signals. Nevertheless, a prevalent drawback of these systems is their incapacity to handle long-tail items and cold-start scenarios. With the recent advent of large language models, there has been a significant increase in research efforts focused on exploiting LLMs to extract semantic information for users and items. However, text-based recommendations highly rely on elaborate feature engineering and frequently fail to capture collaborative similarities. To overcome these limitations, we propose a novel pre-training framework for sequential recommendation, termed PRECISE. This framework combines collaborative signals with semantic information. Moreover, PRECISE employs a learning framework that initially models users' comprehensive interests across all recommendation scenarios and subsequently concentrates on the specific interests of target-scene behaviors. We demonstrate that PRECISE precisely captures the entire range of user interests and effectively transfers them to the target interests. Empirical findings reveal that the PRECISE framework attains outstanding performance on both public and industrial datasets.

Related papers

• RecGPT Technical Report [57.84251629878726]
  We propose RecGPT, a next-generation framework that places user intent at the center of the recommendation pipeline.<n> RecGPT integrates large language models into key stages of user interest mining, item retrieval, and explanation generation.<n>Online experiments demonstrate that RecGPT achieves consistent performance gains across stakeholders.
  arXiv  Detail & Related papers  (2025-07-30T17:55:06Z)
• Thought-Augmented Planning for LLM-Powered Interactive Recommender Agent [56.61028117645315]
  We propose a novel thought-augmented interactive recommender agent system (TAIRA) that addresses complex user intents through distilled thought patterns.<n>Specifically, TAIRA is designed as an LLM-powered multi-agent system featuring a manager agent that orchestrates recommendation tasks by decomposing user needs and planning subtasks.<n>Through comprehensive experiments conducted across multiple datasets, TAIRA exhibits significantly enhanced performance compared to existing methods.
  arXiv  Detail & Related papers  (2025-06-30T03:15:50Z)
• Synergizing Implicit and Explicit User Interests: A Multi-Embedding Retrieval Framework at Pinterest [9.904093205817247]
  The retrieval stage plays a critical role in generating a high-recall set of candidate items.<n>Traditional two-tower models struggle in this regard due to limited user-item feature interaction.<n>We propose a novel multi-embedding retrieval framework designed to enhance user interest representation.
  arXiv  Detail & Related papers  (2025-06-29T02:14:21Z)
• PERSCEN: Learning Personalized Interaction Pattern and Scenario Preference for Multi-Scenario Matching [38.829190984763294]
  Key to effective multi-scenario recommendation lies in capturing both user preferences shared across all scenarios and scenario-aware preferences specific to each scenario.<n>We propose PERSCEN, an innovative approach that incorporates user-specific modeling into multi-scenario matching.<n>PERSCEN constructs a user-specific feature graph based on user characteristics and employs a lightweight graph neural network to capture higher-order interaction patterns.
  arXiv  Detail & Related papers  (2025-06-23T08:15:16Z)
• Multi-agents based User Values Mining for Recommendation [52.26100802380767]
  We propose a zero-shot multi-LLM collaborative framework for effective and accurate user value extraction.<n>We apply text summarization techniques to condense item content while preserving essential meaning.<n>To mitigate hallucinations, we introduce two specialized agent roles: evaluators and supervisors.
  arXiv  Detail & Related papers  (2025-05-02T04:01:31Z)
• Large Language Model Empowered Recommendation Meets All-domain Continual Pre-Training [60.38082979765664]
  CPRec is an All-domain Continual Pre-Training framework for Recommendation. It holistically align LLMs with universal user behaviors through the continual pre-training paradigm. We conduct experiments on five real-world datasets from two distinct platforms.
  arXiv  Detail & Related papers  (2025-04-11T20:01:25Z)
• Modeling Churn in Recommender Systems with Aggregated Preferences [6.261444979025644]
  We propose a model that addresses the dual challenges of leveraging aggregated user information and mitigating churn risk. Our model assumes that the RS operates with a probabilistic prior over user types and aggregated satisfaction levels for various content types.
  arXiv  Detail & Related papers  (2025-02-09T13:12:11Z)
• LLM-assisted Explicit and Implicit Multi-interest Learning Framework for Sequential Recommendation [50.98046887582194]
  We propose an explicit and implicit multi-interest learning framework to model user interests on two levels: behavior and semantics. The proposed EIMF framework effectively and efficiently combines small models with LLM to improve the accuracy of multi-interest modeling.
  arXiv  Detail & Related papers  (2024-11-14T13:00:23Z)
• Retrieval Augmentation via User Interest Clustering [57.63883506013693]
  Industrial recommender systems are sensitive to the patterns of user-item engagement. We propose a novel approach that efficiently constructs user interest and facilitates low computational cost inference. Our approach has been deployed in multiple products at Meta, facilitating short-form video related recommendation.
  arXiv  Detail & Related papers  (2024-08-07T16:35:10Z)
• Knowledge Adaptation from Large Language Model to Recommendation for Practical Industrial Application [54.984348122105516]
  Large Language Models (LLMs) pretrained on massive text corpus presents a promising avenue for enhancing recommender systems. We propose an Llm-driven knowlEdge Adaptive RecommeNdation (LEARN) framework that synergizes open-world knowledge with collaborative knowledge.
  arXiv  Detail & Related papers  (2024-05-07T04:00:30Z)
• Contrastive Learning Method for Sequential Recommendation based on Multi-Intention Disentanglement [5.734747179463411]
  We propose a Contrastive Learning sequential recommendation method based on Multi-Intention Disentanglement (MIDCL) In our work, intentions are recognized as dynamic and diverse, and user behaviors are often driven by current multi-intentions. We propose two types of contrastive learning paradigms for finding the most relevant user's interactive intention, and maximizing the mutual information of positive sample pairs.
  arXiv  Detail & Related papers  (2024-04-28T15:13:36Z)
• Cross-Domain Causal Preference Learning for Out-of-Distribution Recommendation [0.8315707564931466]
  Current recommender systems rely on the assumption that the training and testing datasets have identical distributions. This study proposes a novel model called Cross-Domain Causal Preference Learning for Out-of-Distribution Recommendation (CDCOR)
  arXiv  Detail & Related papers  (2024-04-23T09:21:15Z)
• Large Language Models meet Collaborative Filtering: An Efficient All-round LLM-based Recommender System [19.8986219047121]
  Collaborative filtering recommender systems (CF-RecSys) have shown successive results in enhancing the user experience on social media and e-commerce platforms. Recent strategies have focused on leveraging modality information of user/items based on pre-trained modality encoders and Large Language Models. We propose an efficient All-round LLM-based Recommender system, called A-LLMRec, that excels not only in the cold scenario but also in the warm scenario.
  arXiv  Detail & Related papers  (2024-04-17T13:03:07Z)
• InteraRec: Screenshot Based Recommendations Using Multimodal Large Language Models [0.6926105253992517]
  We introduce a sophisticated and interactive recommendation framework denoted as InteraRec. InteraRec captures high-frequency screenshots of web pages as users navigate through a website. We demonstrate the effectiveness of InteraRec in providing users with valuable and personalized offerings.
  arXiv  Detail & Related papers  (2024-02-26T17:47:57Z)
• MISSRec: Pre-training and Transferring Multi-modal Interest-aware Sequence Representation for Recommendation [61.45986275328629]
  We propose MISSRec, a multi-modal pre-training and transfer learning framework for sequential recommendation. On the user side, we design a Transformer-based encoder-decoder model, where the contextual encoder learns to capture the sequence-level multi-modal user interests. On the candidate item side, we adopt a dynamic fusion module to produce user-adaptive item representation.
  arXiv  Detail & Related papers  (2023-08-22T04:06:56Z)
• Generative Slate Recommendation with Reinforcement Learning [49.75985313698214]
  reinforcement learning algorithms can be used to optimize user engagement in recommender systems. However, RL approaches are intractable in the slate recommendation scenario. In that setting, an action corresponds to a slate that may contain any combination of items. In this work we propose to encode slates in a continuous, low-dimensional latent space learned by a variational auto-encoder. We are able to (i) relax assumptions required by previous work, and (ii) improve the quality of the action selection by modeling full slates.
  arXiv  Detail & Related papers  (2023-01-20T15:28:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.