ADER: Adaptively Distilled Exemplar Replay Towards Continual Learning
for Session-based Recommendation
- URL: http://arxiv.org/abs/2007.12000v1
- Date: Thu, 23 Jul 2020 13:19:53 GMT
- Title: ADER: Adaptively Distilled Exemplar Replay Towards Continual Learning
for Session-based Recommendation
- Authors: Fei Mi, Xiaoyu Lin, and Boi Faltings
- Abstract summary: Session-based recommendation has received growing attention recently due to the increasing privacy concern.
We propose a method called Adaptively Distilled Exemplar Replay (ADER) by periodically replaying previous training samples.
ADER consistently outperforms other baselines, and it even outperforms the method using all historical data at every update cycle.
- Score: 28.22402119581332
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Session-based recommendation has received growing attention recently due to
the increasing privacy concern. Despite the recent success of neural
session-based recommenders, they are typically developed in an offline manner
using a static dataset. However, recommendation requires continual adaptation
to take into account new and obsolete items and users, and requires "continual
learning" in real-life applications. In this case, the recommender is updated
continually and periodically with new data that arrives in each update cycle,
and the updated model needs to provide recommendations for user activities
before the next model update. A major challenge for continual learning with
neural models is catastrophic forgetting, in which a continually trained model
forgets user preference patterns it has learned before. To deal with this
challenge, we propose a method called Adaptively Distilled Exemplar Replay
(ADER) by periodically replaying previous training samples (i.e., exemplars) to
the current model with an adaptive distillation loss. Experiments are conducted
based on the state-of-the-art SASRec model using two widely used datasets to
benchmark ADER with several well-known continual learning techniques. We
empirically demonstrate that ADER consistently outperforms other baselines, and
it even outperforms the method using all historical data at every update cycle.
This result reveals that ADER is a promising solution to mitigate the
catastrophic forgetting issue towards building more realistic and scalable
session-based recommenders.
Related papers
- TTT4Rec: A Test-Time Training Approach for Rapid Adaption in Sequential Recommendation [11.15566809055308]
Test-Time Training (TTT) offers a novel approach by using self-supervised learning during inference to dynamically update model parameters.
We propose TTT4Rec, a sequential recommendation framework that integrates TTT to better capture dynamic user behavior.
We evaluate TTT4Rec on three widely-used recommendation datasets, demonstrating that it achieves performance on par with or exceeding state-of-the-art models.
arXiv Detail & Related papers (2024-09-27T21:14:23Z) - Adaptive Rentention & Correction for Continual Learning [114.5656325514408]
A common problem in continual learning is the classification layer's bias towards the most recent task.
We name our approach Adaptive Retention & Correction (ARC)
ARC achieves an average performance increase of 2.7% and 2.6% on the CIFAR-100 and Imagenet-R datasets.
arXiv Detail & Related papers (2024-05-23T08:43:09Z) - Federated Privacy-preserving Collaborative Filtering for On-Device Next
App Prediction [52.16923290335873]
We propose a novel SeqMF model to solve the problem of predicting the next app launch during mobile device usage.
We modify the structure of the classical matrix factorization model and update the training procedure to sequential learning.
One more ingredient of the proposed approach is a new privacy mechanism that guarantees the protection of the sent data from the users to the remote server.
arXiv Detail & Related papers (2023-02-05T10:29:57Z) - Effective and Efficient Training for Sequential Recommendation using
Recency Sampling [91.02268704681124]
We propose a novel Recency-based Sampling of Sequences training objective.
We show that the models enhanced with our method can achieve performances exceeding or very close to stateof-the-art BERT4Rec.
arXiv Detail & Related papers (2022-07-06T13:06:31Z) - WSLRec: Weakly Supervised Learning for Neural Sequential Recommendation
Models [24.455665093145818]
We propose a novel model-agnostic training approach called WSLRec, which adopts a three-stage framework: pre-training, top-$k$ mining, intrinsic and fine-tuning.
WSLRec resolves the incompleteness problem by pre-training models on extra weak supervisions from model-free methods like BR and ItemCF, while resolving the inaccuracy problem by leveraging the top-$k$ mining to screen out reliable user-item relevance from weak supervisions for fine-tuning.
arXiv Detail & Related papers (2022-02-28T08:55:12Z) - Learning to Learn a Cold-start Sequential Recommender [70.5692886883067]
The cold-start recommendation is an urgent problem in contemporary online applications.
We propose a meta-learning based cold-start sequential recommendation framework called metaCSR.
metaCSR holds the ability to learn the common patterns from regular users' behaviors.
arXiv Detail & Related papers (2021-10-18T08:11:24Z) - Contrastive Self-supervised Sequential Recommendation with Robust
Augmentation [101.25762166231904]
Sequential Recommendationdescribes a set of techniques to model dynamic user behavior in order to predict future interactions in sequential user data.
Old and new issues remain, including data-sparsity and noisy data.
We propose Contrastive Self-Supervised Learning for sequential Recommendation (CoSeRec)
arXiv Detail & Related papers (2021-08-14T07:15:25Z) - Hybrid Model with Time Modeling for Sequential Recommender Systems [0.15229257192293202]
Booking.com organized the WSDM WebTour 2021 Challenge, which aims to benchmark models to recommend the final city in a trip.
We conducted several experiments to test different state-of-the-art deep learning architectures for recommender systems.
Our experimental result shows that the improved NARM outperforms all other state-of-the-art benchmark methods.
arXiv Detail & Related papers (2021-03-07T19:28:22Z) - S^3-Rec: Self-Supervised Learning for Sequential Recommendation with
Mutual Information Maximization [104.87483578308526]
We propose the model S3-Rec, which stands for Self-Supervised learning for Sequential Recommendation.
For our task, we devise four auxiliary self-supervised objectives to learn the correlations among attribute, item, subsequence, and sequence.
Extensive experiments conducted on six real-world datasets demonstrate the superiority of our proposed method over existing state-of-the-art methods.
arXiv Detail & Related papers (2020-08-18T11:44:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.