Performance Comparison of Session-based Recommendation Algorithms based on GNNs

• URL: http://arxiv.org/abs/2312.16695v2
• Date: Thu, 18 Jul 2024 13:02:16 GMT
• Title: Performance Comparison of Session-based Recommendation Algorithms based on GNNs
• Authors: Faisal Shehzad, Dietmar Jannach,
• Abstract summary: In session-based recommendation settings, a recommender system has no access to long-term user profiles. We present the results of an evaluation of eight recent GNN-based approaches that were published in high-quality outlets.
• Score: 6.617487928813376
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In session-based recommendation settings, a recommender system has no access to long-term user profiles and thus has to base its suggestions on the user interactions that are observed in an ongoing session. Since such sessions can consist of only a small set of interactions, various approaches based on Graph Neural Networks (GNN) were recently proposed, as they allow us to integrate various types of side information about the items in a natural way. Unfortunately, a variety of evaluation settings are used in the literature, e.g., in terms of protocols, metrics and baselines, making it difficult to assess what represents the state of the art. In this work, we present the results of an evaluation of eight recent GNN-based approaches that were published in high-quality outlets. For a fair comparison, all models are systematically tuned and tested under identical conditions using three common datasets. We furthermore include k-nearest-neighbor and sequential rules-based models as baselines, as such models have previously exhibited competitive performance results for similar settings. To our surprise, the evaluation showed that the simple models outperform all recent GNN models in terms of the Mean Reciprocal Rank, which we used as an optimization criterion, and were only outperformed in three cases in terms of the Hit Rate. Additional analyses furthermore reveal that several other factors that are often not deeply discussed in papers, e.g., random seeds, can markedly impact the performance of GNN-based models. Our results therefore (a) point to continuing issues in the community in terms of research methodology and (b) indicate that there is ample room for improvement in session-based recommendation.

Related papers

• A Novel Evaluation Perspective on GNNs-based Recommender Systems through the Topology of the User-Item Graph [14.12873271435375]
  Graph neural networks (GNNs)-based recommender systems have encountered great success in recommendation. We provide a novel evaluation perspective on GNNs-based recommendation, which investigates the impact of the graph topology on the recommendation performance.
  arXiv  Detail & Related papers  (2024-08-21T16:34:53Z)
• A GNN Model with Adaptive Weights for Session-Based Recommendation Systems [0.0]
  We present a novel approach that can be used in session-based recommendations (SBRs) We introduce an adaptive weighting mechanism applied to the graph neural network (GNN) vectors. Items are assigned varying degrees of importance within each session as a result of the weighting mechanism.
  arXiv  Detail & Related papers  (2024-08-09T13:13:43Z)
• Exploring Popularity Bias in Session-based Recommendation [0.6798775532273751]
  We extend the analysis to session-based setup and adapted propensity calculation to the unique characteristics of session-based recommendation tasks. We study the distributions of propensity and different stratification techniques on different datasets and find that propensity-related traits are actually dataset-specific.
  arXiv  Detail & Related papers  (2023-12-13T02:48:35Z)
• Evaluating Graph Neural Networks for Link Prediction: Current Pitfalls and New Benchmarking [66.83273589348758]
  Link prediction attempts to predict whether an unseen edge exists based on only a portion of edges of a graph. A flurry of methods have been introduced in recent years that attempt to make use of graph neural networks (GNNs) for this task. New and diverse datasets have also been created to better evaluate the effectiveness of these new models.
  arXiv  Detail & Related papers  (2023-06-18T01:58:59Z)
• On Pitfalls of Test-Time Adaptation [82.8392232222119]
  Test-Time Adaptation (TTA) has emerged as a promising approach for tackling the robustness challenge under distribution shifts. We present TTAB, a test-time adaptation benchmark that encompasses ten state-of-the-art algorithms, a diverse array of distribution shifts, and two evaluation protocols.
  arXiv  Detail & Related papers  (2023-06-06T09:35:29Z)
• Ordinal Graph Gamma Belief Network for Social Recommender Systems [54.9487910312535]
  We develop a hierarchical Bayesian model termed ordinal graph factor analysis (OGFA), which jointly models user-item and user-user interactions. OGFA not only achieves good recommendation performance, but also extracts interpretable latent factors corresponding to representative user preferences. We extend OGFA to ordinal graph gamma belief network, which is a multi-stochastic-layer deep probabilistic model.
  arXiv  Detail & Related papers  (2022-09-12T09:19:22Z)
• Neighbor Enhanced Graph Convolutional Networks for Node Classification and Recommendation [30.179717374489414]
  We theoretically analyze the affection of the neighbor quality over GCN models' performance. We propose the Neighbor Enhanced Graph Convolutional Network (NEGCN) framework to boost the performance of existing GCN models.
  arXiv  Detail & Related papers  (2022-03-30T06:54:28Z)
• Three Steps to Multimodal Trajectory Prediction: Modality Clustering, Classification and Synthesis [54.249502356251085]
  We present a novel insight along with a brand-new prediction framework. Our proposed method surpasses state-of-the-art works even without introducing social and map information.
  arXiv  Detail & Related papers  (2021-03-14T06:21:03Z)
• Few-Shot Named Entity Recognition: A Comprehensive Study [92.40991050806544]
  We investigate three schemes to improve the model generalization ability for few-shot settings. We perform empirical comparisons on 10 public NER datasets with various proportions of labeled data. We create new state-of-the-art results on both few-shot and training-free settings.
  arXiv  Detail & Related papers  (2020-12-29T23:43:16Z)
• Single-Layer Graph Convolutional Networks For Recommendation [17.3621098912528]
  Graph Convolutional Networks (GCNs) have received significant attention and achieved start-of-the-art performances on recommendation tasks. Existing GCN models tend to perform recursion aggregations among all related nodes, which arises severe computational burden. We propose a single GCN layer to aggregate information from the neighbors filtered by DA similarity and then generates the node representations.
  arXiv  Detail & Related papers  (2020-06-07T14:38:47Z)
• Meta-Learned Confidence for Few-shot Learning [60.6086305523402]
  A popular transductive inference technique for few-shot metric-based approaches, is to update the prototype of each class with the mean of the most confident query examples. We propose to meta-learn the confidence for each query sample, to assign optimal weights to unlabeled queries. We validate our few-shot learning model with meta-learned confidence on four benchmark datasets.
  arXiv  Detail & Related papers  (2020-02-27T10:22:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.