Contextual Distillation Model for Diversified Recommendation

• URL: http://arxiv.org/abs/2406.09021v2
• Date: Wed, 14 Aug 2024 04:52:02 GMT
• Title: Contextual Distillation Model for Diversified Recommendation
• Authors: Fan Li, Xu Si, Shisong Tang, Dingmin Wang, Kunyan Han, Bing Han, Guorui Zhou, Yang Song, Hechang Chen,
• Abstract summary: Contextual Distillation Model (CDM) is an efficient recommendation model that addresses diversification. We propose a contrastive context encoder that employs attention mechanisms to model both positive and negative contexts. During inference, ranking is performed through a linear combination of the recommendation and student model scores.
• Score: 19.136439564988834
• License:
• Abstract: The diversity of recommendation is equally crucial as accuracy in improving user experience. Existing studies, e.g., Determinantal Point Process (DPP) and Maximal Marginal Relevance (MMR), employ a greedy paradigm to iteratively select items that optimize both accuracy and diversity. However, prior methods typically exhibit quadratic complexity, limiting their applications to the re-ranking stage and are not applicable to other recommendation stages with a larger pool of candidate items, such as the pre-ranking and ranking stages. In this paper, we propose Contextual Distillation Model (CDM), an efficient recommendation model that addresses diversification, suitable for the deployment in all stages of industrial recommendation pipelines. Specifically, CDM utilizes the candidate items in the same user request as context to enhance the diversification of the results. We propose a contrastive context encoder that employs attention mechanisms to model both positive and negative contexts. For the training of CDM, we compare each target item with its context embedding and utilize the knowledge distillation framework to learn the win probability of each target item under the MMR algorithm, where the teacher is derived from MMR outputs. During inference, ranking is performed through a linear combination of the recommendation and student model scores, ensuring both diversity and efficiency. We perform offline evaluations on two industrial datasets and conduct online A/B test of CDM on the short-video platform KuaiShou. The considerable enhancements observed in both recommendation quality and diversity, as shown by metrics, provide strong superiority for the effectiveness of CDM.

Related papers

• Learning k-Determinantal Point Processes for Personalized Ranking [13.677246792673564]
  We present a new optimization criterion LkP based on set probability comparison for personalized ranking. LkP is broadly applicable, and when applied to existing recommendation models it also yields strong performance improvements.
  arXiv  Detail & Related papers  (2024-06-23T02:24:50Z)
• Are We Really Achieving Better Beyond-Accuracy Performance in Next Basket Recommendation? [57.91114305844153]
  Next basket recommendation (NBR) is a special type of sequential recommendation that is increasingly receiving attention. Recent studies into NBR have found a substantial performance difference between recommending repeat items and explore items. We propose a plug-and-play two-step repetition-exploration framework that treats repeat items and explores items separately.
  arXiv  Detail & Related papers  (2024-05-02T09:59:35Z)
• Curriculum-scheduled Knowledge Distillation from Multiple Pre-trained Teachers for Multi-domain Sequential Recommendation [102.91236882045021]
  It is essential to explore how to use different pre-trained recommendation models efficiently in real-world systems. We propose a novel curriculum-scheduled knowledge distillation from multiple pre-trained teachers for multi-domain sequential recommendation. CKD-MDSR takes full advantages of different PRMs as multiple teacher models to boost a small student recommendation model.
  arXiv  Detail & Related papers  (2024-01-01T15:57:15Z)
• Diversely Regularized Matrix Factorization for Accurate and Aggregately Diversified Recommendation [15.483426620593013]
  DivMF (Diversely Regularized Matrix Factorization) is a novel matrix factorization method for aggregately diversified recommendation. We show that DivMF achieves the state-of-the-art performance in aggregately diversified recommendation.
  arXiv  Detail & Related papers  (2022-10-19T08:49:39Z)
• Recommendation Systems with Distribution-Free Reliability Guarantees [83.80644194980042]
  We show how to return a set of items rigorously guaranteed to contain mostly good items. Our procedure endows any ranking model with rigorous finite-sample control of the false discovery rate. We evaluate our methods on the Yahoo! Learning to Rank and MSMarco datasets.
  arXiv  Detail & Related papers  (2022-07-04T17:49:25Z)
• Choosing the Best of Both Worlds: Diverse and Novel Recommendations through Multi-Objective Reinforcement Learning [68.45370492516531]
  We introduce Scalarized Multi-Objective Reinforcement Learning (SMORL) for the Recommender Systems (RS) setting. SMORL agent augments standard recommendation models with additional RL layers that enforce it to simultaneously satisfy three principal objectives: accuracy, diversity, and novelty of recommendations. Our experimental results on two real-world datasets reveal a substantial increase in aggregate diversity, a moderate increase in accuracy, reduced repetitiveness of recommendations, and demonstrate the importance of reinforcing diversity and novelty as complementary objectives.
  arXiv  Detail & Related papers  (2021-10-28T13:22:45Z)
• Distiller: A Systematic Study of Model Distillation Methods in Natural Language Processing [21.215122347801696]
  We aim to identify how different components in the KD pipeline affect the resulting performance. We propose Distiller, a meta KD framework that combines a broad range of techniques across different stages of the KD pipeline. We find that different datasets/tasks prefer different KD algorithms, and thus propose a simple AutoDistiller algorithm.
  arXiv  Detail & Related papers  (2021-09-23T02:12:28Z)
• Reenvisioning Collaborative Filtering vs Matrix Factorization [65.74881520196762]
  Collaborative filtering models based on matrix factorization and learned similarities using Artificial Neural Networks (ANNs) have gained significant attention in recent years. Announcement of ANNs within the recommendation ecosystem has been recently questioned, raising several comparisons in terms of efficiency and effectiveness. We show the potential these techniques may have on beyond-accuracy evaluation while analyzing effect on complementary evaluation dimensions.
  arXiv  Detail & Related papers  (2021-07-28T16:29:38Z)
• Sample-Rank: Weak Multi-Objective Recommendations Using Rejection Sampling [0.5156484100374059]
  We introduce a method involving multi-goal sampling followed by ranking for user-relevance (Sample-Rank) to nudge recommendations towards multi-objective goals of the marketplace. The proposed method's novelty is that it reduces the MO recommendation problem to sampling from a desired multi-goal distribution then using it to build a production-friendly learning-to-rank model.
  arXiv  Detail & Related papers  (2020-08-24T09:17:18Z)
• Sequential Recommendation with Self-Attentive Multi-Adversarial Network [101.25533520688654]
  We present a Multi-Factor Generative Adversarial Network (MFGAN) for explicitly modeling the effect of context information on sequential recommendation. Our framework is flexible to incorporate multiple kinds of factor information, and is able to trace how each factor contributes to the recommendation decision over time.
  arXiv  Detail & Related papers  (2020-05-21T12:28:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.