Sequential Recommendation on Temporal Proximities with Contrastive Learning and Self-Attention

• URL: http://arxiv.org/abs/2402.09784v2
• Date: Sun, 18 Feb 2024 02:38:02 GMT
• Title: Sequential Recommendation on Temporal Proximities with Contrastive Learning and Self-Attention
• Authors: Hansol Jung, Hyunwoo Seo and Chiehyeon Lim
• Abstract summary: Sequential recommender systems identify user preferences from their past interactions to predict subsequent items optimally. Recent models often neglect similarities in users' actions that occur implicitly among users during analogous timeframes. We propose a sequential recommendation model called TemProxRec, which includes contrastive learning and self-attention methods to consider temporal proximities.
• Score: 3.7182810519704095
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sequential recommender systems identify user preferences from their past interactions to predict subsequent items optimally. Although traditional deep-learning-based models and modern transformer-based models in previous studies capture unidirectional and bidirectional patterns within user-item interactions, the importance of temporal contexts, such as individual behavioral and societal trend patterns, remains underexplored. Notably, recent models often neglect similarities in users' actions that occur implicitly among users during analogous timeframes-a concept we term vertical temporal proximity. These models primarily adapt the self-attention mechanisms of the transformer to consider the temporal context in individual user actions. Meanwhile, this adaptation still remains limited in considering the horizontal temporal proximity within item interactions, like distinguishing between subsequent item purchases within a week versus a month. To address these gaps, we propose a sequential recommendation model called TemProxRec, which includes contrastive learning and self-attention methods to consider temporal proximities both across and within user-item interactions. The proposed contrastive learning method learns representations of items selected in close temporal periods across different users to be close. Simultaneously, the proposed self-attention mechanism encodes temporal and positional contexts in a user sequence using both absolute and relative embeddings. This way, our TemProxRec accurately predicts the relevant items based on the user-item interactions within a specific timeframe. We validate this work through comprehensive experiments on TemProxRec, consistently outperforming existing models on benchmark datasets as well as showing the significance of considering the vertical and horizontal temporal proximities into sequential recommendation.

Related papers

• TimeSiam: A Pre-Training Framework for Siamese Time-Series Modeling [67.02157180089573]
  Time series pre-training has recently garnered wide attention for its potential to reduce labeling expenses and benefit various downstream tasks. This paper proposes TimeSiam as a simple but effective self-supervised pre-training framework for Time series based on Siamese networks.
  arXiv  Detail & Related papers  (2024-02-04T13:10:51Z)
• Multi-Behavior Sequential Recommendation with Temporal Graph Transformer [66.10169268762014]
  We tackle the dynamic user-item relation learning with the awareness of multi-behavior interactive patterns. We propose a new Temporal Graph Transformer (TGT) recommendation framework to jointly capture dynamic short-term and long-range user-item interactive patterns.
  arXiv  Detail & Related papers  (2022-06-06T15:42:54Z)
• Continuous-Time User Preference Modelling for Temporal Sets Prediction [32.35733523016208]
  We propose a continuous-time user preference modelling framework for temporal sets prediction. We first construct a universal sequence by arranging all the user-set interactions in a non-descending temporal order. For each interaction, we continuously update the memories of the related user and elements based on their currently encoded messages and past memories.
  arXiv  Detail & Related papers  (2022-04-12T02:49:27Z)
• Modeling Dynamic User Preference via Dictionary Learning for Sequential Recommendation [133.8758914874593]
  Capturing the dynamics in user preference is crucial to better predict user future behaviors because user preferences often drift over time. Many existing recommendation algorithms -- including both shallow and deep ones -- often model such dynamics independently. This paper considers the problem of embedding a user's sequential behavior into the latent space of user preferences.
  arXiv  Detail & Related papers  (2022-04-02T03:23:46Z)
• Sequential Recommendation via Stochastic Self-Attention [68.52192964559829]
  Transformer-based approaches embed items as vectors and use dot-product self-attention to measure the relationship between items. We propose a novel textbfSTOchastic textbfSelf-textbfAttention(STOSA) to overcome these issues. We devise a novel Wasserstein Self-Attention module to characterize item-item position-wise relationships in sequences.
  arXiv  Detail & Related papers  (2022-01-16T12:38:45Z)
• TEA: A Sequential Recommendation Framework via Temporally Evolving Aggregations [12.626079984394766]
  We propose a novel sequential recommendation framework based on dynamic user-item heterogeneous graphs. We exploit the conditional random field to aggregate the heterogeneous graphs and user behaviors for probability estimation. We provide scalable and flexible implementations of the proposed framework.
  arXiv  Detail & Related papers  (2021-11-14T15:54:23Z)
• Learning Dual Dynamic Representations on Time-Sliced User-Item Interaction Graphs for Sequential Recommendation [62.30552176649873]
  We devise a novel Dynamic Representation Learning model for Sequential Recommendation (DRL-SRe) To better model the user-item interactions for characterizing the dynamics from both sides, the proposed model builds a global user-item interaction graph for each time slice. To enable the model to capture fine-grained temporal information, we propose an auxiliary temporal prediction task over consecutive time slices.
  arXiv  Detail & Related papers  (2021-09-24T07:44:27Z)
• Learning Heterogeneous Temporal Patterns of User Preference for Timely Recommendation [15.930016839929047]
  We propose a novel recommender system for timely recommendations, called TimelyRec. In TimelyRec, a cascade of two encoders captures the temporal patterns of user preference using a proposed attention module for each encoder. Our experiments on a scenario for item recommendation and the proposed scenario for item-timing recommendation on real-world datasets demonstrate the superiority of TimelyRec.
  arXiv  Detail & Related papers  (2021-04-29T08:37:30Z)
• Dynamic Embeddings for Interaction Prediction [2.5758502140236024]
  In recommender systems (RSs), predicting the next item that a user interacts with is critical for user retention. Recent studies have shown the effectiveness of modeling the mutual interactions between users and items using separate user and item embeddings. We propose a novel method called DeePRed that addresses some of their limitations.
  arXiv  Detail & Related papers  (2020-11-10T16:04:46Z)
• MEANTIME: Mixture of Attention Mechanisms with Multi-temporal Embeddings for Sequential Recommendation [12.386304516106854]
  Self-attention based models have achieved state-of-the-art performance in sequential recommendation task. These models rely on a simple positional embedding to exploit the sequential nature of the user's history. We propose MEANTIME which employs multiple types of temporal embeddings designed to capture various patterns from the user's behavior sequence.
  arXiv  Detail & Related papers  (2020-08-19T05:32:14Z)
• Predicting Temporal Sets with Deep Neural Networks [50.53727580527024]
  We propose an integrated solution based on the deep neural networks for temporal sets prediction. A unique perspective is to learn element relationship by constructing set-level co-occurrence graph. We design an attention-based module to adaptively learn the temporal dependency of elements and sets.
  arXiv  Detail & Related papers  (2020-06-20T03:29:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.