Effective and Efficient Training for Sequential Recommendation using Recency Sampling

• URL: http://arxiv.org/abs/2207.02643v1
• Date: Wed, 6 Jul 2022 13:06:31 GMT
• Title: Effective and Efficient Training for Sequential Recommendation using Recency Sampling
• Authors: Aleksandr Petrov and Craig Macdonald
• Abstract summary: We propose a novel Recency-based Sampling of Sequences training objective. We show that the models enhanced with our method can achieve performances exceeding or very close to stateof-the-art BERT4Rec.
• Score: 91.02268704681124
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many modern sequential recommender systems use deep neural networks, which can effectively estimate the relevance of items but require a lot of time to train. Slow training increases expenses, hinders product development timescales and prevents the model from being regularly updated to adapt to changing user preferences. Training such sequential models involves appropriately sampling past user interactions to create a realistic training objective. The existing training objectives have limitations. For instance, next item prediction never uses the beginning of the sequence as a learning target, thereby potentially discarding valuable data. On the other hand, the item masking used by BERT4Rec is only weakly related to the goal of the sequential recommendation; therefore, it requires much more time to obtain an effective model. Hence, we propose a novel Recency-based Sampling of Sequences training objective that addresses both limitations. We apply our method to various recent and state-of-the-art model architectures - such as GRU4Rec, Caser, and SASRec. We show that the models enhanced with our method can achieve performances exceeding or very close to stateof-the-art BERT4Rec, but with much less training time.

Related papers

• Always-Sparse Training by Growing Connections with Guided Stochastic Exploration [46.4179239171213]
  We propose an efficient always-sparse training algorithm with excellent scaling to larger and sparser models. We evaluate our method on CIFAR-10/100 and ImageNet using VGG, and ViT models, and compare it against a range of sparsification methods.
  arXiv  Detail & Related papers  (2024-01-12T21:32:04Z)
• Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization [51.34904967046097]
  Continual learning seeks to overcome the challenge of catastrophic forgetting, where a model forgets previously learnt information. We introduce a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting. Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments.
  arXiv  Detail & Related papers  (2023-09-15T17:10:51Z)
• PILOT: A Pre-Trained Model-Based Continual Learning Toolbox [71.63186089279218]
  This paper introduces a pre-trained model-based continual learning toolbox known as PILOT. On the one hand, PILOT implements some state-of-the-art class-incremental learning algorithms based on pre-trained models, such as L2P, DualPrompt, and CODA-Prompt. On the other hand, PILOT fits typical class-incremental learning algorithms within the context of pre-trained models to evaluate their effectiveness.
  arXiv  Detail & Related papers  (2023-09-13T17:55:11Z)
• Towards More Robust and Accurate Sequential Recommendation with Cascade-guided Adversarial Training [54.56998723843911]
  Two properties unique to the nature of sequential recommendation models may impair their robustness. We propose Cascade-guided Adversarial training, a new adversarial training procedure that is specifically designed for sequential recommendation models.
  arXiv  Detail & Related papers  (2023-04-11T20:55:02Z)
• Intersection of Parallels as an Early Stopping Criterion [64.8387564654474]
  We propose a method to spot an early stopping point in the training iterations without the need for a validation set. For a wide range of learning rates, our method, called Cosine-Distance Criterion (CDC), leads to better generalization on average than all the methods that we compare against.
  arXiv  Detail & Related papers  (2022-08-19T19:42:41Z)
• WSLRec: Weakly Supervised Learning for Neural Sequential Recommendation Models [24.455665093145818]
  We propose a novel model-agnostic training approach called WSLRec, which adopts a three-stage framework: pre-training, top-$k$ mining, intrinsic and fine-tuning. WSLRec resolves the incompleteness problem by pre-training models on extra weak supervisions from model-free methods like BR and ItemCF, while resolving the inaccuracy problem by leveraging the top-$k$ mining to screen out reliable user-item relevance from weak supervisions for fine-tuning.
  arXiv  Detail & Related papers  (2022-02-28T08:55:12Z)
• Deep Ensembles for Low-Data Transfer Learning [21.578470914935938]
  We study different ways of creating ensembles from pre-trained models. We show that the nature of pre-training itself is a performant source of diversity. We propose a practical algorithm that efficiently identifies a subset of pre-trained models for any downstream dataset.
  arXiv  Detail & Related papers  (2020-10-14T07:59:00Z)
• A Practical Incremental Method to Train Deep CTR Models [37.54660958085938]
  We introduce a practical incremental method to train deep CTR models, which consists of three decoupled modules. Our method can achieve comparable performance to the conventional batch mode training with much better training efficiency.
  arXiv  Detail & Related papers  (2020-09-04T12:35:42Z)
• S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization [104.87483578308526]
  We propose the model S3-Rec, which stands for Self-Supervised learning for Sequential Recommendation. For our task, we devise four auxiliary self-supervised objectives to learn the correlations among attribute, item, subsequence, and sequence. Extensive experiments conducted on six real-world datasets demonstrate the superiority of our proposed method over existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-08-18T11:44:10Z)
• ADER: Adaptively Distilled Exemplar Replay Towards Continual Learning for Session-based Recommendation [28.22402119581332]
  Session-based recommendation has received growing attention recently due to the increasing privacy concern. We propose a method called Adaptively Distilled Exemplar Replay (ADER) by periodically replaying previous training samples. ADER consistently outperforms other baselines, and it even outperforms the method using all historical data at every update cycle.
  arXiv  Detail & Related papers  (2020-07-23T13:19:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.