Pre-trained Neural Recommenders: A Transferable Zero-Shot Framework for Recommendation Systems

• URL: http://arxiv.org/abs/2309.01188v2
• Date: Fri, 29 Sep 2023 15:54:33 GMT
• Title: Pre-trained Neural Recommenders: A Transferable Zero-Shot Framework for Recommendation Systems
• Authors: Junting Wang, Adit Krishnan, Hari Sundaram, Yunzhe Li
• Abstract summary: We show how to learn universal (i.e., supporting zero-shot adaptation without user or item auxiliary information) representations for nodes and edges from the bipartite user-item interaction graph.
• Score: 5.597511654202054
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern neural collaborative filtering techniques are critical to the success of e-commerce, social media, and content-sharing platforms. However, despite technical advances -- for every new application domain, we need to train an NCF model from scratch. In contrast, pre-trained vision and language models are routinely applied to diverse applications directly (zero-shot) or with limited fine-tuning. Inspired by the impact of pre-trained models, we explore the possibility of pre-trained recommender models that support building recommender systems in new domains, with minimal or no retraining, without the use of any auxiliary user or item information. Zero-shot recommendation without auxiliary information is challenging because we cannot form associations between users and items across datasets when there are no overlapping users or items. Our fundamental insight is that the statistical characteristics of the user-item interaction matrix are universally available across different domains and datasets. Thus, we use the statistical characteristics of the user-item interaction matrix to identify dataset-independent representations for users and items. We show how to learn universal (i.e., supporting zero-shot adaptation without user or item auxiliary information) representations for nodes and edges from the bipartite user-item interaction graph. We learn representations by exploiting the statistical properties of the interaction data, including user and item marginals, and the size and density distributions of their clusters.

Related papers

• LLM-assisted Explicit and Implicit Multi-interest Learning Framework for Sequential Recommendation [50.98046887582194]
  We propose an explicit and implicit multi-interest learning framework to model user interests on two levels: behavior and semantics. The proposed EIMF framework effectively and efficiently combines small models with LLM to improve the accuracy of multi-interest modeling.
  arXiv  Detail & Related papers  (2024-11-14T13:00:23Z)
• Retrieval Augmentation via User Interest Clustering [57.63883506013693]
  Industrial recommender systems are sensitive to the patterns of user-item engagement. We propose a novel approach that efficiently constructs user interest and facilitates low computational cost inference. Our approach has been deployed in multiple products at Meta, facilitating short-form video related recommendation.
  arXiv  Detail & Related papers  (2024-08-07T16:35:10Z)
• Cross-Attribute Matrix Factorization Model with Shared User Embedding [0.5266869303483376]
  We introduce a refined NeuMF model that considers not only the interaction between users and items, but also acrossing associated attributes. Our proposed architecture features a shared user embedding, seamlessly integrating with user embeddings to imporve the robustness and effectively address the cold-start problem.
  arXiv  Detail & Related papers  (2023-08-14T17:15:37Z)
• Client-specific Property Inference against Secure Aggregation in Federated Learning [52.8564467292226]
  Federated learning has become a widely used paradigm for collaboratively training a common model among different participants. Many attacks have shown that it is still possible to infer sensitive information such as membership, property, or outright reconstruction of participant data. We show that simple linear models can effectively capture client-specific properties only from the aggregated model updates.
  arXiv  Detail & Related papers  (2023-03-07T14:11:01Z)
• Intent-aware Multi-source Contrastive Alignment for Tag-enhanced Recommendation [46.04494053005958]
  We seek an alternative framework that is light and effective through self-supervised learning across different sources of information. We use a self-supervision signal to pair users with the auxiliary information associated with the items they have interacted with before. We show that our method can achieve better performance while requiring less training time.
  arXiv  Detail & Related papers  (2022-11-11T17:43:19Z)
• Ordinal Graph Gamma Belief Network for Social Recommender Systems [54.9487910312535]
  We develop a hierarchical Bayesian model termed ordinal graph factor analysis (OGFA), which jointly models user-item and user-user interactions. OGFA not only achieves good recommendation performance, but also extracts interpretable latent factors corresponding to representative user preferences. We extend OGFA to ordinal graph gamma belief network, which is a multi-stochastic-layer deep probabilistic model.
  arXiv  Detail & Related papers  (2022-09-12T09:19:22Z)
• IA-GCN: Interactive Graph Convolutional Network for Recommendation [13.207235494649343]
  Graph Convolutional Network (GCN) has become a novel state-of-the-art for Collaborative Filtering (CF) based Recommender Systems (RS) We build bilateral interactive guidance between each user-item pair and propose a new model named IA-GCN (short for InterActive GCN) Our model is built on top of LightGCN, a state-of-the-art GCN model for CF, and can be combined with various GCN-based CF architectures in an end-to-end fashion.
  arXiv  Detail & Related papers  (2022-04-08T03:38:09Z)
• Knowledge-Enhanced Hierarchical Graph Transformer Network for Multi-Behavior Recommendation [56.12499090935242]
  This work proposes a Knowledge-Enhanced Hierarchical Graph Transformer Network (KHGT) to investigate multi-typed interactive patterns between users and items in recommender systems. KHGT is built upon a graph-structured neural architecture to capture type-specific behavior characteristics. We show that KHGT consistently outperforms many state-of-the-art recommendation methods across various evaluation settings.
  arXiv  Detail & Related papers  (2021-10-08T09:44:00Z)
• Learning Dual Dynamic Representations on Time-Sliced User-Item Interaction Graphs for Sequential Recommendation [62.30552176649873]
  We devise a novel Dynamic Representation Learning model for Sequential Recommendation (DRL-SRe) To better model the user-item interactions for characterizing the dynamics from both sides, the proposed model builds a global user-item interaction graph for each time slice. To enable the model to capture fine-grained temporal information, we propose an auxiliary temporal prediction task over consecutive time slices.
  arXiv  Detail & Related papers  (2021-09-24T07:44:27Z)
• Exploiting Behavioral Consistence for Universal User Representation [11.290137806288191]
  We focus on developing universal user representation model. The obtained universal representations are expected to contain rich information. We propose Self-supervised User Modeling Network (SUMN) to encode behavior data into the universal representation.
  arXiv  Detail & Related papers  (2020-12-11T06:10:14Z)
• Personalized Federated Learning: A Meta-Learning Approach [28.281166755509886]
  In Federated Learning, we aim to train models across multiple computing units (users) In this paper, we study a personalized variant of the federated learning in which our goal is to find an initial shared model that current or new users can easily adapt to their local dataset by performing one or a few steps of gradient descent with respect to their own data.
  arXiv  Detail & Related papers  (2020-02-19T01:08:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.