Inductive Generative Recommendation via Retrieval-based Speculation

• URL: http://arxiv.org/abs/2410.02939v1
• Date: Thu, 03 Oct 2024 19:32:32 GMT
• Title: Inductive Generative Recommendation via Retrieval-based Speculation
• Abstract summary: Generative recommendation (GR) is an emerging paradigm that tokenizes items into discrete tokens and learns to autoregressively generate the next tokens as predictions. In this paper, we propose SpecGR, a plug-and-play framework that enables GR models to recommend new items in an inductive setting.
• Score: 26.70518822003545
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative recommendation (GR) is an emerging paradigm that tokenizes items into discrete tokens and learns to autoregressively generate the next tokens as predictions. Although effective, GR models operate in a transductive setting, meaning they can only generate items seen during training without applying heuristic re-ranking strategies. In this paper, we propose SpecGR, a plug-and-play framework that enables GR models to recommend new items in an inductive setting. SpecGR uses a drafter model with inductive capability to propose candidate items, which may include both existing items and new items. The GR model then acts as a verifier, accepting or rejecting candidates while retaining its strong ranking capabilities. We further introduce the guided re-drafting technique to make the proposed candidates more aligned with the outputs of generative recommendation models, improving the verification efficiency. We consider two variants for drafting: (1) using an auxiliary drafter model for better flexibility, or (2) leveraging the GR model's own encoder for parameter-efficient self-drafting. Extensive experiments on three real-world datasets demonstrate that SpecGR exhibits both strong inductive recommendation ability and the best overall performance among the compared methods. Our code is available at: https://github.com/Jamesding000/SpecGR.

Related papers

• Structuring and Tokenizing Distributed User Interest Context for Generative Recommendation [60.03046892817763]
  Generative recommendation is an emerging paradigm that has shown promise in industrial recommendation systems.<n>Existing methods often struggle to organize and inject complex user-behavioral and item-semantic contexts into recommendation models simultaneously.<n>We propose G2Rec, a scalable framework that unifies holistic graph-based user co-engagement modeling with semantic tokenization for industrial-scale generative recommendation.
  arXiv  Detail & Related papers  (2026-06-18T17:58:32Z)
• RAGR: Review-Augmented Generative Recommendation [61.29879991713178]
  Review-Augmented Generative Recommendation (RAGR) is a novel GR framework that incorporates review feedback directly into the generative user sequence.<n>RAGR yields consistent and significant gains over strong GR backbones across all metrics.
  arXiv  Detail & Related papers  (2026-05-17T05:21:23Z)
• On the Equivalence Between Auto-Regressive Next Token Prediction and Full-Item-Vocabulary Maximum Likelihood Estimation in Generative Recommendation--A Short Note [13.85356879537006]
  Generative recommendation (GR) has emerged as a widely adopted paradigm in sequential recommendation.<n>Current GR systems follow a similar pipeline: tokenization for item indexing, next-token prediction as the training objective and auto-regressive decoding for next-item generation.
  arXiv  Detail & Related papers  (2026-04-17T06:27:42Z)
• SimGR: Escaping the Pitfalls of Generative Decoding in LLM-based Recommendation [68.00727783181289]
  A core objective in recommender systems is to accurately model the distribution of user preferences over items to enable personalized recommendations.<n>We observe that existing methods inevitably introduce systematic bias when estimating item-level preference distributions.<n>We propose textbfSimply textbfGenerative textbfRecommendation (textbfSimGR), a framework that directly models item-level preference distributions in a shared latent space.
  arXiv  Detail & Related papers  (2026-02-08T07:26:52Z)
• GRPO-RM: Fine-Tuning Representation Models via GRPO-Driven Reinforcement Learning [52.16150076582931]
  We propose Group Relative Policy Optimization for Representation Model (GRPO-RM)<n>Our method establishes a predefined output set to functionally replace token sequence sampling in large language models (LLMs)<n>A specialized reward function is designed to accommodate the properties of representation models.
  arXiv  Detail & Related papers  (2025-11-19T09:19:39Z)
• OneRec-Think: In-Text Reasoning for Generative Recommendation [55.53292983432484]
  OneRec-Think is a unified framework that seamlessly integrates dialogue, reasoning, and personalized recommendation.<n>Our proposed "Think-Ahead" architecture enables effective industrial deployment on Kuaishou, achieving a 0.159% gain in APP Stay Time.
  arXiv  Detail & Related papers  (2025-10-13T17:20:13Z)
• Do We Really Need Specialization? Evaluating Generalist Text Embeddings for Zero-Shot Recommendation and Search [51.30225948318648]
  Generalist Text Embedding Models (GTEs) are pre-trained on large-scale corpora.<n>We show that GTEs can guarantee strong zero-shot performance without specialized adaptation.<n>We also show that compressing embedding dimensions by focusing on the most informative directions effectively reduces noise.
  arXiv  Detail & Related papers  (2025-07-07T13:41:52Z)
• GFlowGR: Fine-tuning Generative Recommendation Frameworks with Generative Flow Networks [36.39484385717512]
  This paper treats the Generative Recommendations (GR) as a multi-step generation task and constructs a GFlowNets-based fine-tuning framework (GFlowGR)<n>The proposed framework integrates collaborative knowledge from traditional recommender systems to create an adaptive trajectory sampler and a comprehensive reward model.
  arXiv  Detail & Related papers  (2025-06-19T08:04:31Z)
• LARES: Latent Reasoning for Sequential Recommendation [96.26996622771593]
  We present LARES, a novel and scalable LAtent REasoning framework for Sequential recommendation.<n>Our proposed approach employs a recurrent architecture that allows flexible expansion of reasoning depth without increasing parameter complexity.<n>Our framework exhibits seamless compatibility with existing advanced models, further improving their recommendation performance.
  arXiv  Detail & Related papers  (2025-05-22T16:22:54Z)
• PinRec: Outcome-Conditioned, Multi-Token Generative Retrieval for Industry-Scale Recommendation Systems [6.738040775297612]
  This paper introduces PinRec, a novel generative retrieval model developed for applications at Pinterest. Our experiments demonstrate that PinRec can successfully balance performance, diversity, and efficiency, delivering a significant positive impact to users.
  arXiv  Detail & Related papers  (2025-04-09T17:46:12Z)
• OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignment [9.99840965933561]
  We propose OneRec, which replaces the cascaded learning framework with a unified generative model. OneRec includes: 1) an encoder-decoder structure, which encodes the user's historical behavior sequences and gradually decodes the videos that the user may be interested in.
  arXiv  Detail & Related papers  (2025-02-26T09:25:10Z)
• One2set + Large Language Model: Best Partners for Keyphrase Generation [42.969689556605005]
  Keyphrase generation (KPG) aims to automatically generate a collection of phrases representing the core concepts of a given document. We introduce a generate-then-select framework decomposing KPG into two steps, where we adopt a one2set-based model as generator to produce candidates and then use an LLM as selector to select keyphrases from these candidates. Our framework significantly surpasses state-of-the-art models, especially in absent keyphrase prediction.
  arXiv  Detail & Related papers  (2024-10-04T13:31:09Z)
• Language Models are Graph Learners [70.14063765424012]
  Language Models (LMs) are challenging the dominance of domain-specific models, including Graph Neural Networks (GNNs) and Graph Transformers (GTs) We propose a novel approach that empowers off-the-shelf LMs to achieve performance comparable to state-of-the-art GNNs on node classification tasks.
  arXiv  Detail & Related papers  (2024-10-03T08:27:54Z)
• Generative Recommender with End-to-End Learnable Item Tokenization [51.82768744368208]
  We introduce ETEGRec, a novel End-To-End Generative Recommender that unifies item tokenization and generative recommendation into a cohesive framework.<n>ETEGRec consists of an item tokenizer and a generative recommender built on a dual encoder-decoder architecture.<n>We develop an alternating optimization technique to ensure stable and efficient end-to-end training of the entire framework.
  arXiv  Detail & Related papers  (2024-09-09T12:11:53Z)
• GenRec: Generative Sequential Recommendation with Large Language Models [4.381277509913139]
  We propose a novel model named Generative Recommendation (GenRec) GenRec is lightweight and requires only a few hours to train effectively in low-resource settings. Our experiments have demonstrated that GenRec generalizes on various public real-world datasets.
  arXiv  Detail & Related papers  (2024-07-30T20:58:36Z)
• Distillation Enhanced Generative Retrieval [96.69326099136289]
  Generative retrieval is a promising new paradigm in text retrieval that generates identifier strings of relevant passages as the retrieval target. In this work, we identify a viable direction to further enhance generative retrieval via distillation and propose a feasible framework, named DGR. We conduct experiments on four public datasets, and the results indicate that DGR achieves state-of-the-art performance among the generative retrieval methods.
  arXiv  Detail & Related papers  (2024-02-16T15:48:24Z)
• LightLM: A Lightweight Deep and Narrow Language Model for Generative Recommendation [45.00339682494516]
  LightLM is a lightweight Transformer-based language model for generative recommendation. LightLM tackles the issue by introducing a light-weight deep and narrow Transformer architecture. We also show that our devised user and item ID indexing methods, i.e., Spectral Collaborative Indexing (SCI) and Graph Collaborative Indexing (GCI), enables the deep and narrow Transformer architecture to outperform large-scale language models for recommendation.
  arXiv  Detail & Related papers  (2023-10-26T15:44:57Z)
• Large Language Models are Zero-Shot Rankers for Recommender Systems [76.02500186203929]
  This work aims to investigate the capacity of large language models (LLMs) to act as the ranking model for recommender systems. We show that LLMs have promising zero-shot ranking abilities but struggle to perceive the order of historical interactions. We demonstrate that these issues can be alleviated using specially designed prompting and bootstrapping strategies.
  arXiv  Detail & Related papers  (2023-05-15T17:57:39Z)
• Guiding Generative Language Models for Data Augmentation in Few-Shot Text Classification [59.698811329287174]
  We leverage GPT-2 for generating artificial training instances in order to improve classification performance. Our results show that fine-tuning GPT-2 in a handful of label instances leads to consistent classification improvements.
  arXiv  Detail & Related papers  (2021-11-17T12:10:03Z)
• Few-Shot Named Entity Recognition: A Comprehensive Study [92.40991050806544]
  We investigate three schemes to improve the model generalization ability for few-shot settings. We perform empirical comparisons on 10 public NER datasets with various proportions of labeled data. We create new state-of-the-art results on both few-shot and training-free settings.
  arXiv  Detail & Related papers  (2020-12-29T23:43:16Z)
• Sequential Recommendation with Self-Attentive Multi-Adversarial Network [101.25533520688654]
  We present a Multi-Factor Generative Adversarial Network (MFGAN) for explicitly modeling the effect of context information on sequential recommendation. Our framework is flexible to incorporate multiple kinds of factor information, and is able to trace how each factor contributes to the recommendation decision over time.
  arXiv  Detail & Related papers  (2020-05-21T12:28:59Z)
• Interpretable Learning-to-Rank with Generalized Additive Models [78.42800966500374]
  Interpretability of learning-to-rank models is a crucial yet relatively under-examined research area. Recent progress on interpretable ranking models largely focuses on generating post-hoc explanations for existing black-box ranking models. We lay the groundwork for intrinsically interpretable learning-to-rank by introducing generalized additive models (GAMs) into ranking tasks.
  arXiv  Detail & Related papers  (2020-05-06T01:51:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.