Optimizing Encoder-Only Transformers for Session-Based Recommendation Systems

• URL: http://arxiv.org/abs/2410.11150v1
• Date: Tue, 15 Oct 2024 00:23:18 GMT
• Title: Optimizing Encoder-Only Transformers for Session-Based Recommendation Systems
• Authors: Anis Redjdal, Luis Pinto, Michel Desmarais,
• Abstract summary: We introduce Sequential Masked Modeling to tackle the challenges of single-session recommendation. Our method combines data augmentation through window sliding with a unique penultimate token masking strategy to capture sequential dependencies more effectively. We evaluate our approach on three widely-used datasets, Yoochoose 1/64, Diginetica, and Tmall.
• Score: 0.0
• License:
• Abstract: Session-based recommendation is the task of predicting the next item a user will interact with, often without access to historical user data. In this work, we introduce Sequential Masked Modeling, a novel approach for encoder-only transformer architectures to tackle the challenges of single-session recommendation. Our method combines data augmentation through window sliding with a unique penultimate token masking strategy to capture sequential dependencies more effectively. By enhancing how transformers handle session data, Sequential Masked Modeling significantly improves next-item prediction performance. We evaluate our approach on three widely-used datasets, Yoochoose 1/64, Diginetica, and Tmall, comparing it to state-of-the-art single-session, cross-session, and multi-relation approaches. The results demonstrate that our Transformer-SMM models consistently outperform all models that rely on the same amount of information, while even rivaling methods that have access to more extensive user history. This study highlights the potential of encoder-only transformers in session-based recommendation and opens the door for further improvements.

Related papers

• Sample Enrichment via Temporary Operations on Subsequences for Sequential Recommendation [15.718287580146272]
  We propose a novel model-agnostic and highly generic framework for sequential recommendation called sample enrichment via temporary operations on subsequences (SETO) We highlight our SETO's effectiveness and versatility over multiple representative and state-of-the-art sequential recommendation models across multiple real-world datasets.
  arXiv  Detail & Related papers  (2024-07-25T06:22:08Z)
• Pareto Front Approximation for Multi-Objective Session-Based Recommender Systems [0.0]
  MultiTRON is an approach that adapts approximation techniques to multi-objective session-based recommender systems. Our approach optimize trade-offs between key metrics such as click-through and conversion rates by training on sampled preference vectors. We validate the model's performance through extensive offline and online evaluation.
  arXiv  Detail & Related papers  (2024-07-23T20:38:23Z)
• MMGRec: Multimodal Generative Recommendation with Transformer Model [81.61896141495144]
  MMGRec aims to introduce a generative paradigm into multimodal recommendation. We first devise a hierarchical quantization method Graph CF-RQVAE to assign Rec-ID for each item from its multimodal information. We then train a Transformer-based recommender to generate the Rec-IDs of user-preferred items based on historical interaction sequences.
  arXiv  Detail & Related papers  (2024-04-25T12:11:27Z)
• Enhancing User Intent Capture in Session-Based Recommendation with Attribute Patterns [77.19390850643944]
  We propose the Frequent Attribute Pattern Augmented Transformer (FAPAT) FAPAT characterizes user intents by building attribute transition graphs and matching attribute patterns. We demonstrate that FAPAT consistently outperforms state-of-the-art methods by an average of 4.5% across various evaluation metrics.
  arXiv  Detail & Related papers  (2023-12-23T03:28:18Z)
• MISSRec: Pre-training and Transferring Multi-modal Interest-aware Sequence Representation for Recommendation [61.45986275328629]
  We propose MISSRec, a multi-modal pre-training and transfer learning framework for sequential recommendation. On the user side, we design a Transformer-based encoder-decoder model, where the contextual encoder learns to capture the sequence-level multi-modal user interests. On the candidate item side, we adopt a dynamic fusion module to produce user-adaptive item representation.
  arXiv  Detail & Related papers  (2023-08-22T04:06:56Z)
• ClusterSeq: Enhancing Sequential Recommender Systems with Clustering based Meta-Learning [3.168790535780547]
  ClusterSeq is a Meta-Learning Clustering-Based Sequential Recommender System. It exploits dynamic information in the user sequence to enhance item prediction accuracy, even in the absence of side information. Our proposed approach achieves a substantial improvement of 16-39% in Mean Reciprocal Rank (MRR)
  arXiv  Detail & Related papers  (2023-07-25T18:53:24Z)
• Federated Privacy-preserving Collaborative Filtering for On-Device Next App Prediction [52.16923290335873]
  We propose a novel SeqMF model to solve the problem of predicting the next app launch during mobile device usage. We modify the structure of the classical matrix factorization model and update the training procedure to sequential learning. One more ingredient of the proposed approach is a new privacy mechanism that guarantees the protection of the sent data from the users to the remote server.
  arXiv  Detail & Related papers  (2023-02-05T10:29:57Z)
• Masked Autoencoding for Scalable and Generalizable Decision Making [93.84855114717062]
  MaskDP is a simple and scalable self-supervised pretraining method for reinforcement learning and behavioral cloning. We find that a MaskDP model gains the capability of zero-shot transfer to new BC tasks, such as single and multiple goal reaching.
  arXiv  Detail & Related papers  (2022-11-23T07:04:41Z)
• Exploiting Session Information in BERT-based Session-aware Sequential Recommendation [13.15762859612114]
  In recommendation systems, utilizing the user interaction history as sequential information has resulted in great performance improvement. In many online services, user interactions are commonly grouped by sessions that presumably share preferences, which requires a different approach from ordinary sequence representation techniques. We propose three methods to improve recommendation performance by exploiting session information while minimizing additional parameters in a BERT-based sequential recommendation model.
  arXiv  Detail & Related papers  (2022-04-22T17:58:10Z)
• Contrastive Self-supervised Sequential Recommendation with Robust Augmentation [101.25762166231904]
  Sequential Recommendationdescribes a set of techniques to model dynamic user behavior in order to predict future interactions in sequential user data. Old and new issues remain, including data-sparsity and noisy data. We propose Contrastive Self-Supervised Learning for sequential Recommendation (CoSeRec)
  arXiv  Detail & Related papers  (2021-08-14T07:15:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.