Does It Look Sequential? An Analysis of Datasets for Evaluation of Sequential Recommendations

• URL: http://arxiv.org/abs/2408.12008v1
• Date: Wed, 21 Aug 2024 21:40:07 GMT
• Title: Does It Look Sequential? An Analysis of Datasets for Evaluation of Sequential Recommendations
• Authors: Anton Klenitskiy, Anna Volodkevich, Anton Pembek, Alexey Vasilev,
• Abstract summary: Sequential recommender systems aim to use the order of interactions in a user's history to predict future interactions. It is crucial to use datasets that exhibit a sequential structure to evaluate sequential recommenders properly. We apply several methods based on the random shuffling of the user's sequence of interactions to assess the strength of sequential structure across 15 datasets.
• Score: 0.8437187555622164
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sequential recommender systems are an important and demanded area of research. Such systems aim to use the order of interactions in a user's history to predict future interactions. The premise is that the order of interactions and sequential patterns play an essential role. Therefore, it is crucial to use datasets that exhibit a sequential structure to evaluate sequential recommenders properly. We apply several methods based on the random shuffling of the user's sequence of interactions to assess the strength of sequential structure across 15 datasets, frequently used for sequential recommender systems evaluation in recent research papers presented at top-tier conferences. As shuffling explicitly breaks sequential dependencies inherent in datasets, we estimate the strength of sequential patterns by comparing metrics for shuffled and original versions of the dataset. Our findings show that several popular datasets have a rather weak sequential structure.

Related papers

• Capturing Temporal Components for Time Series Classification [5.70772577110828]
  This work introduces a textitcompositional representation learning approach trained on statistically coherent components extracted from sequential data. Based on a multi-scale change space, an unsupervised approach is proposed to segment the sequential data into chunks with similar statistical properties. A sequence-based encoder model is trained in a multi-task setting to learn compositional representations from these temporal components for time series classification.
  arXiv  Detail & Related papers  (2024-06-20T16:15:21Z)
• Retrieving Continuous Time Event Sequences using Neural Temporal Point Processes with Learnable Hashing [24.963828650935913]
  We propose NeuroSeqRet, a first-of-its-kind framework designed specifically for end-to-end CTES retrieval. We develop four variants of the relevance model for different kinds of applications based on the trade-off between accuracy and efficiency. Our experiments show the significant accuracy boost of NeuroSeqRet as well as the efficacy of our hashing mechanism.
  arXiv  Detail & Related papers  (2023-07-13T18:54:50Z)
• Recommender Systems with Generative Retrieval [58.454606442670034]
  We propose a novel generative retrieval approach, where the retrieval model autoregressively decodes the identifiers of the target candidates. To that end, we create semantically meaningful of codewords to serve as a Semantic ID for each item. We show that recommender systems trained with the proposed paradigm significantly outperform the current SOTA models on various datasets.
  arXiv  Detail & Related papers  (2023-05-08T21:48:17Z)
• Towards Out-of-Distribution Sequential Event Prediction: A Causal Treatment [72.50906475214457]
  The goal of sequential event prediction is to estimate the next event based on a sequence of historical events. In practice, the next-event prediction models are trained with sequential data collected at one time. We propose a framework with hierarchical branching structures for learning context-specific representations.
  arXiv  Detail & Related papers  (2022-10-24T07:54:13Z)
• Sequential Modelling with Applications to Music Recommendation, Fact-Checking, and Speed Reading [4.434614653851092]
  This thesis makes methodological contributions and new investigations of sequential modelling for the specific application areas of systems that recommend music tracks to listeners and systems that process text semantics.
  arXiv  Detail & Related papers  (2021-09-11T08:05:48Z)
• Contrastive Self-supervised Sequential Recommendation with Robust Augmentation [101.25762166231904]
  Sequential Recommendationdescribes a set of techniques to model dynamic user behavior in order to predict future interactions in sequential user data. Old and new issues remain, including data-sparsity and noisy data. We propose Contrastive Self-Supervised Learning for sequential Recommendation (CoSeRec)
  arXiv  Detail & Related papers  (2021-08-14T07:15:25Z)
• Sequence Adaptation via Reinforcement Learning in Recommender Systems [8.909115457491522]
  We propose the SAR model, which learns the sequential patterns and adjusts the sequence length of user-item interactions in a personalized manner. In addition, we optimize a joint loss function to align the accuracy of the sequential recommendations with the expected cumulative rewards of the critic network. Our experimental evaluation on four real-world datasets demonstrates the superiority of our proposed model over several baseline approaches.
  arXiv  Detail & Related papers  (2021-07-31T13:56:46Z)
• Dynamic Memory based Attention Network for Sequential Recommendation [79.5901228623551]
  We propose a novel long sequential recommendation model called Dynamic Memory-based Attention Network (DMAN) It segments the overall long behavior sequence into a series of sub-sequences, then trains the model and maintains a set of memory blocks to preserve long-term interests of users. Based on the dynamic memory, the user's short-term and long-term interests can be explicitly extracted and combined for efficient joint recommendation.
  arXiv  Detail & Related papers  (2021-02-18T11:08:54Z)
• S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization [104.87483578308526]
  We propose the model S3-Rec, which stands for Self-Supervised learning for Sequential Recommendation. For our task, we devise four auxiliary self-supervised objectives to learn the correlations among attribute, item, subsequence, and sequence. Extensive experiments conducted on six real-world datasets demonstrate the superiority of our proposed method over existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-08-18T11:44:10Z)
• Sequential recommendation with metric models based on frequent sequences [0.688204255655161]
  We propose to use frequent sequences to identify the most relevant part of the user history for the recommendation. The most salient items are then used in a unified metric model that embeds items based on user preferences and sequential dynamics.
  arXiv  Detail & Related papers  (2020-08-12T22:08:04Z)
• Document Ranking with a Pretrained Sequence-to-Sequence Model [56.44269917346376]
  We show how a sequence-to-sequence model can be trained to generate relevance labels as "target words" Our approach significantly outperforms an encoder-only model in a data-poor regime.
  arXiv  Detail & Related papers  (2020-03-14T22:29:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.