Performance Comparison of Session-based Recommendation Algorithms based on GNNs
- URL: http://arxiv.org/abs/2312.16695v2
- Date: Thu, 18 Jul 2024 13:02:16 GMT
- Title: Performance Comparison of Session-based Recommendation Algorithms based on GNNs
- Authors: Faisal Shehzad, Dietmar Jannach,
- Abstract summary: In session-based recommendation settings, a recommender system has no access to long-term user profiles.
We present the results of an evaluation of eight recent GNN-based approaches that were published in high-quality outlets.
- Score: 6.617487928813376
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In session-based recommendation settings, a recommender system has no access to long-term user profiles and thus has to base its suggestions on the user interactions that are observed in an ongoing session. Since such sessions can consist of only a small set of interactions, various approaches based on Graph Neural Networks (GNN) were recently proposed, as they allow us to integrate various types of side information about the items in a natural way. Unfortunately, a variety of evaluation settings are used in the literature, e.g., in terms of protocols, metrics and baselines, making it difficult to assess what represents the state of the art. In this work, we present the results of an evaluation of eight recent GNN-based approaches that were published in high-quality outlets. For a fair comparison, all models are systematically tuned and tested under identical conditions using three common datasets. We furthermore include k-nearest-neighbor and sequential rules-based models as baselines, as such models have previously exhibited competitive performance results for similar settings. To our surprise, the evaluation showed that the simple models outperform all recent GNN models in terms of the Mean Reciprocal Rank, which we used as an optimization criterion, and were only outperformed in three cases in terms of the Hit Rate. Additional analyses furthermore reveal that several other factors that are often not deeply discussed in papers, e.g., random seeds, can markedly impact the performance of GNN-based models. Our results therefore (a) point to continuing issues in the community in terms of research methodology and (b) indicate that there is ample room for improvement in session-based recommendation.
Related papers
- Exploring Popularity Bias in Session-based Recommendation [0.6798775532273751]
We extend the analysis to session-based setup and adapted propensity calculation to the unique characteristics of session-based recommendation tasks.
We study the distributions of propensity and different stratification techniques on different datasets and find that propensity-related traits are actually dataset-specific.
arXiv Detail & Related papers (2023-12-13T02:48:35Z) - Evaluating Graph Neural Networks for Link Prediction: Current Pitfalls
and New Benchmarking [66.83273589348758]
Link prediction attempts to predict whether an unseen edge exists based on only a portion of edges of a graph.
A flurry of methods have been introduced in recent years that attempt to make use of graph neural networks (GNNs) for this task.
New and diverse datasets have also been created to better evaluate the effectiveness of these new models.
arXiv Detail & Related papers (2023-06-18T01:58:59Z) - On Pitfalls of Test-Time Adaptation [82.8392232222119]
Test-Time Adaptation (TTA) has emerged as a promising approach for tackling the robustness challenge under distribution shifts.
We present TTAB, a test-time adaptation benchmark that encompasses ten state-of-the-art algorithms, a diverse array of distribution shifts, and two evaluation protocols.
arXiv Detail & Related papers (2023-06-06T09:35:29Z) - Flattened Graph Convolutional Networks For Recommendation [18.198536511983452]
This paper proposes the flattened GCN(FlatGCN) model, which is able to achieve superior performance with remarkably less complexity compared with existing models.
First, we propose a simplified but powerful GCN architecture which aggregates the neighborhood information using one flattened GCN layer.
Second, we propose an informative neighbor-infomax sampling method to select the most valuable neighbors by measuring the correlation among neighboring nodes.
Third, we propose a layer ensemble technique which improves the expressiveness of the learned representations by assembling the layer-wise neighborhood representations at the final layer.
arXiv Detail & Related papers (2022-09-25T12:53:50Z) - Ordinal Graph Gamma Belief Network for Social Recommender Systems [54.9487910312535]
We develop a hierarchical Bayesian model termed ordinal graph factor analysis (OGFA), which jointly models user-item and user-user interactions.
OGFA not only achieves good recommendation performance, but also extracts interpretable latent factors corresponding to representative user preferences.
We extend OGFA to ordinal graph gamma belief network, which is a multi-stochastic-layer deep probabilistic model.
arXiv Detail & Related papers (2022-09-12T09:19:22Z) - Neighbor Enhanced Graph Convolutional Networks for Node Classification
and Recommendation [30.179717374489414]
We theoretically analyze the affection of the neighbor quality over GCN models' performance.
We propose the Neighbor Enhanced Graph Convolutional Network (NEGCN) framework to boost the performance of existing GCN models.
arXiv Detail & Related papers (2022-03-30T06:54:28Z) - Improved Representation Learning for Session-based Recommendation [0.0]
Session-based recommendation systems suggest relevant items to users by modeling user behavior and preferences using short-term anonymous sessions.
Existing methods leverage Graph Neural Networks (GNNs) that propagate and aggregate information from neighboring nodes.
We propose using a Transformer in combination with a target attentive GNN, which allows richer Representation Learning.
arXiv Detail & Related papers (2021-07-04T00:57:28Z) - Three Steps to Multimodal Trajectory Prediction: Modality Clustering,
Classification and Synthesis [54.249502356251085]
We present a novel insight along with a brand-new prediction framework.
Our proposed method surpasses state-of-the-art works even without introducing social and map information.
arXiv Detail & Related papers (2021-03-14T06:21:03Z) - Few-Shot Named Entity Recognition: A Comprehensive Study [92.40991050806544]
We investigate three schemes to improve the model generalization ability for few-shot settings.
We perform empirical comparisons on 10 public NER datasets with various proportions of labeled data.
We create new state-of-the-art results on both few-shot and training-free settings.
arXiv Detail & Related papers (2020-12-29T23:43:16Z) - Single-Layer Graph Convolutional Networks For Recommendation [17.3621098912528]
Graph Convolutional Networks (GCNs) have received significant attention and achieved start-of-the-art performances on recommendation tasks.
Existing GCN models tend to perform recursion aggregations among all related nodes, which arises severe computational burden.
We propose a single GCN layer to aggregate information from the neighbors filtered by DA similarity and then generates the node representations.
arXiv Detail & Related papers (2020-06-07T14:38:47Z) - Meta-Learned Confidence for Few-shot Learning [60.6086305523402]
A popular transductive inference technique for few-shot metric-based approaches, is to update the prototype of each class with the mean of the most confident query examples.
We propose to meta-learn the confidence for each query sample, to assign optimal weights to unlabeled queries.
We validate our few-shot learning model with meta-learned confidence on four benchmark datasets.
arXiv Detail & Related papers (2020-02-27T10:22:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.