Inductive Generative Recommendation via Retrieval-based Speculation

• URL: http://arxiv.org/abs/2410.02939v1
• Date: Thu, 03 Oct 2024 19:32:32 GMT
• Title: Inductive Generative Recommendation via Retrieval-based Speculation
• Authors: Yijie Ding, Yupeng Hou, Jiacheng Li, Julian McAuley,
• Abstract summary: Generative recommendation (GR) is an emerging paradigm that tokenizes items into discrete tokens and learns to autoregressively generate the next tokens as predictions. In this paper, we propose SpecGR, a plug-and-play framework that enables GR models to recommend new items in an inductive setting.
• Score: 26.70518822003545
• License:
• Abstract: Generative recommendation (GR) is an emerging paradigm that tokenizes items into discrete tokens and learns to autoregressively generate the next tokens as predictions. Although effective, GR models operate in a transductive setting, meaning they can only generate items seen during training without applying heuristic re-ranking strategies. In this paper, we propose SpecGR, a plug-and-play framework that enables GR models to recommend new items in an inductive setting. SpecGR uses a drafter model with inductive capability to propose candidate items, which may include both existing items and new items. The GR model then acts as a verifier, accepting or rejecting candidates while retaining its strong ranking capabilities. We further introduce the guided re-drafting technique to make the proposed candidates more aligned with the outputs of generative recommendation models, improving the verification efficiency. We consider two variants for drafting: (1) using an auxiliary drafter model for better flexibility, or (2) leveraging the GR model's own encoder for parameter-efficient self-drafting. Extensive experiments on three real-world datasets demonstrate that SpecGR exhibits both strong inductive recommendation ability and the best overall performance among the compared methods. Our code is available at: https://github.com/Jamesding000/SpecGR.

Related papers

• One2set + Large Language Model: Best Partners for Keyphrase Generation [42.969689556605005]
  Keyphrase generation (KPG) aims to automatically generate a collection of phrases representing the core concepts of a given document. We introduce a generate-then-select framework decomposing KPG into two steps, where we adopt a one2set-based model as generator to produce candidates and then use an LLM as selector to select keyphrases from these candidates. Our framework significantly surpasses state-of-the-art models, especially in absent keyphrase prediction.
  arXiv  Detail & Related papers  (2024-10-04T13:31:09Z)
• Language Models are Graph Learners [70.14063765424012]
  Language Models (LMs) are challenging the dominance of domain-specific models, including Graph Neural Networks (GNNs) and Graph Transformers (GTs) We propose a novel approach that empowers off-the-shelf LMs to achieve performance comparable to state-of-the-art GNNs on node classification tasks.
  arXiv  Detail & Related papers  (2024-10-03T08:27:54Z)
• GenRec: Generative Sequential Recommendation with Large Language Models [4.381277509913139]
  We propose a novel model named Generative Recommendation (GenRec) GenRec is lightweight and requires only a few hours to train effectively in low-resource settings. Our experiments have demonstrated that GenRec generalizes on various public real-world datasets.
  arXiv  Detail & Related papers  (2024-07-30T20:58:36Z)
• Distillation Enhanced Generative Retrieval [96.69326099136289]
  Generative retrieval is a promising new paradigm in text retrieval that generates identifier strings of relevant passages as the retrieval target. In this work, we identify a viable direction to further enhance generative retrieval via distillation and propose a feasible framework, named DGR. We conduct experiments on four public datasets, and the results indicate that DGR achieves state-of-the-art performance among the generative retrieval methods.
  arXiv  Detail & Related papers  (2024-02-16T15:48:24Z)
• LightLM: A Lightweight Deep and Narrow Language Model for Generative Recommendation [45.00339682494516]
  LightLM is a lightweight Transformer-based language model for generative recommendation. LightLM tackles the issue by introducing a light-weight deep and narrow Transformer architecture. We also show that our devised user and item ID indexing methods, i.e., Spectral Collaborative Indexing (SCI) and Graph Collaborative Indexing (GCI), enables the deep and narrow Transformer architecture to outperform large-scale language models for recommendation.
  arXiv  Detail & Related papers  (2023-10-26T15:44:57Z)
• Large Language Models are Zero-Shot Rankers for Recommender Systems [76.02500186203929]
  This work aims to investigate the capacity of large language models (LLMs) to act as the ranking model for recommender systems. We show that LLMs have promising zero-shot ranking abilities but struggle to perceive the order of historical interactions. We demonstrate that these issues can be alleviated using specially designed prompting and bootstrapping strategies.
  arXiv  Detail & Related papers  (2023-05-15T17:57:39Z)
• Few-Shot Named Entity Recognition: A Comprehensive Study [92.40991050806544]
  We investigate three schemes to improve the model generalization ability for few-shot settings. We perform empirical comparisons on 10 public NER datasets with various proportions of labeled data. We create new state-of-the-art results on both few-shot and training-free settings.
  arXiv  Detail & Related papers  (2020-12-29T23:43:16Z)
• Sequential Recommendation with Self-Attentive Multi-Adversarial Network [101.25533520688654]
  We present a Multi-Factor Generative Adversarial Network (MFGAN) for explicitly modeling the effect of context information on sequential recommendation. Our framework is flexible to incorporate multiple kinds of factor information, and is able to trace how each factor contributes to the recommendation decision over time.
  arXiv  Detail & Related papers  (2020-05-21T12:28:59Z)
• Interpretable Learning-to-Rank with Generalized Additive Models [78.42800966500374]
  Interpretability of learning-to-rank models is a crucial yet relatively under-examined research area. Recent progress on interpretable ranking models largely focuses on generating post-hoc explanations for existing black-box ranking models. We lay the groundwork for intrinsically interpretable learning-to-rank by introducing generalized additive models (GAMs) into ranking tasks.
  arXiv  Detail & Related papers  (2020-05-06T01:51:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.