Differentiable Semantic ID for Generative Recommendation

• URL: http://arxiv.org/abs/2601.19711v1
• Date: Tue, 27 Jan 2026 15:34:11 GMT
• Title: Differentiable Semantic ID for Generative Recommendation
• Abstract summary: Generative recommendation provides a novel paradigm in which each item is represented by a discrete semantic ID (SID) learned from rich content.<n>In practice, SIDs are typically optimized only for content reconstruction rather than recommendation accuracy.<n>A natural approach is to make semantic indexing differentiable so that recommendation gradients can directly influence SID learning.<n>We propose DIGER, a first step toward effective differentiable semantic IDs for generative recommendation.
• Score: 65.83703273297492
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative recommendation provides a novel paradigm in which each item is represented by a discrete semantic ID (SID) learned from rich content. Most existing methods treat SIDs as predefined and train recommenders under static indexing. In practice, SIDs are typically optimized only for content reconstruction rather than recommendation accuracy. This leads to an objective mismatch: the system optimizes an indexing loss to learn the SID and a recommendation loss for interaction prediction, but because the tokenizer is trained independently, the recommendation loss cannot update it. A natural approach is to make semantic indexing differentiable so that recommendation gradients can directly influence SID learning, but this often causes codebook collapse, where only a few codes are used. We attribute this issue to early deterministic assignments that limit codebook exploration, resulting in imbalance and unstable optimization. In this paper, we propose DIGER (Differentiable Semantic ID for Generative Recommendation), a first step toward effective differentiable semantic IDs for generative recommendation. DIGER introduces Gumbel noise to explicitly encourage early-stage exploration over codes, mitigating codebook collapse and improving code utilization. To balance exploration and convergence, we further design two uncertainty decay strategies that gradually reduce the Gumbel noise, enabling a smooth transition from early exploration to exploitation of learned SIDs. Extensive experiments on multiple public datasets demonstrate consistent improvements from differentiable semantic IDs. These results confirm the effectiveness of aligning indexing and recommendation objectives through differentiable SIDs and highlight differentiable semantic indexing as a promising research direction.

Related papers

• Grevo: A Unified Generative Recommendation Framework with Evolutionary Item Indexing [7.05169171974845]
  We propose a unified Generative recommendation framework with evolutionary item indexing.<n>Grevo builds on a single recommender that unifies a behavioral SID generation task and a semantic SID grounding task.<n>We show that Grevo consistently outperforms state-of-the-art generative recommendation methods.
  arXiv  Detail & Related papers  (2026-07-28T06:27:33Z)
• CogRec: Structure-Cognitive Fast-and-Slow Reasoning for Generative Recommendation [29.663400191042996]
  CogRec is a structure-cognitive fast-and-slow reasoning framework.<n>CogRec augments the vertical SID hierarchy with intra-layer semantic graphs and item-level neighborhoods.<n> SID Routing represents recommendation reasoning through layer-wise Match, LateralJump, and Explore operations.
  arXiv  Detail & Related papers  (2026-07-27T13:19:55Z)
• DREAM: Dynamic Refinement of Early Assignment Mappings [7.781421530363403]
  Current methods assign each item a single static identifier through offline tokenization before sufficient user feedback is observed.<n>We identify this one-shot commitment as the fundamental cold-start bottleneck in SID-based generative recommendation.<n>We propose DREAM, a three-stage framework that resolves this flaw through progressive refinement.
  arXiv  Detail & Related papers  (2026-06-05T06:21:01Z)
• SAPO: Step-Aligned Policy Optimization for Reasoning-Based Generative Recommendation [43.98709365191658]
  Generative recommendation treats next-item prediction as autoregressive item-identifier generation.<n>Recent work augments this paradigm with reasoning traces and optimize them via reinforcement learning with verifiable rewards.<n>In large-catalog recommendation, exact-match feedback on the generated SID only reports whether the final item is correct.
  arXiv  Detail & Related papers  (2026-05-17T20:53:08Z)
• Beyond Static Collision Handling: Adaptive Semantic ID Learning for Multimodal Recommendation at Industrial Scale [20.066503602593958]
  AdaSID regulates SID overlaps through a two-stage process.<n>It relaxes repulsion for observed overlaps when the involved items are semantically compatible.<n>It allocates the remaining regulation pressure according to local collision load and training progress.
  arXiv  Detail & Related papers  (2026-04-26T03:59:35Z)
• Reasoning over Semantic IDs Enhances Generative Recommendation [56.83043583765322]
  We propose a two-stage framework that elicits reasoning over SIDs by strengthening SID--language alignment.<n> SIDReasoner further improves recommendation reasoning through outcome-driven reinforced optimization.<n>Our experiments on three real-world datasets demonstrate the effectiveness of our reasoning-augmented SID-based generative recommendation.
  arXiv  Detail & Related papers  (2026-03-24T13:31:48Z)
• APAO: Adaptive Prefix-Aware Optimization for Generative Recommendation [26.371939617653084]
  Generative recommendation is an autoregressive generation process, predicting discrete tokens of the next item conditioned on user interaction histories.<n>Existing generative recommendation models are typically trained with token-level likelihood objectives, such as cross-entropy loss.<n>This leads to a training-inference inconsistency: standard training assumes ground-truth history is always available, ignoring the fact that beam search prunes low-probability branches during inference.
  arXiv  Detail & Related papers  (2026-03-03T08:29:15Z)
• End-to-End Semantic ID Generation for Generative Advertisement Recommendation [33.453121305193434]
  We propose a Unified SID generation framework for generative advertisement recommendation.<n>Specifically, we jointly optimize embeddings and SIDs in an end-to-end manner from raw advertising data.<n>Experiments demonstrate that UniSID consistently outperforms state-of-the-art SID generation methods.
  arXiv  Detail & Related papers  (2026-02-11T02:38:26Z)
• GLASS: A Generative Recommender for Long-sequence Modeling via SID-Tier and Semantic Search [51.44490997013772]
  GLASS is a novel framework that integrates long-term user interests into the generative process via SID-Tier and Semantic Search.<n>We show that GLASS outperforms state-of-the-art baselines in experiments on two large-scale real-world datasets.
  arXiv  Detail & Related papers  (2026-02-05T13:48:33Z)
• Rethinking Generative Recommender Tokenizer: Recsys-Native Encoding and Semantic Quantization Beyond LLMs [17.944727019161878]
  ReSID is a principled, SID framework that recommend learning from the perspective of information preservation and sequential predictability.<n>It consistently outperforms strong sequential and SID-based generative baselines by an average of over 10%, while reducing tokenization cost by up to 122x.
  arXiv  Detail & Related papers  (2026-02-02T17:00:04Z)
• UniGRec: Unified Generative Recommendation with Soft Identifiers for End-to-End Optimization [20.538589808672963]
  We propose a unified generative recommendation framework, UniGRec.<n>UniGRec addresses training-inference discrepancy, item identifier collapse from codeword usage, and collaborative signal deficiency.<n>Experiments on real-world datasets demonstrate that UniGRec consistently outperforms state-of-the-art baseline methods.
  arXiv  Detail & Related papers  (2026-01-24T12:20:29Z)
• TeaRAG: A Token-Efficient Agentic Retrieval-Augmented Generation Framework [62.66056331998838]
  TeaRAG is a token-efficient agentic RAG framework capable of compressing both retrieval content and reasoning steps.<n>Our reward function evaluates the knowledge sufficiency by a knowledge matching mechanism, while penalizing excessive reasoning steps.
  arXiv  Detail & Related papers  (2025-11-07T16:08:34Z)
• Purely Semantic Indexing for LLM-based Generative Recommendation and Retrieval [28.366331215978445]
  We propose purely semantic indexing to generate unique, semantic-preserving IDs without appending non-semantic tokens.<n>We enable unique ID assignment by relaxing the strict nearest-centroid selection and introduce two model-agnostic algorithms.
  arXiv  Detail & Related papers  (2025-09-19T21:59:55Z)
• Exploiting Discriminative Codebook Prior for Autoregressive Image Generation [54.14166700058777]
  token-based autoregressive image generation systems first tokenize images into sequences of token indices with a codebook, and then model these sequences in an autoregressive paradigm.<n>While autoregressive generative models are trained only on index values, the prior encoded in the codebook, which contains rich token similarity information, is not exploited.<n>Recent studies have attempted to incorporate this prior by performing naive k-means clustering on the tokens, helping to facilitate the training of generative models with a reduced codebook.<n>We propose the Discriminative Codebook Prior Extractor (DCPE) as an alternative to k-means
  arXiv  Detail & Related papers  (2025-08-14T15:00:00Z)
• HiD-VAE: Interpretable Generative Recommendation via Hierarchical and Disentangled Semantic IDs [29.735089231891305]
  HiD-VAE is a novel framework that learns hierarchically disentangled item representations through two core innovations.<n>First, HiD-VAE pioneers a hierarchically-supervised quantization process that aligns discrete codes with multi-level item tags.<n>Second, to combat representation entanglement, HiD-VAE incorporates a novel uniqueness loss that directly penalizes latent space overlap.
  arXiv  Detail & Related papers  (2025-08-06T16:45:05Z)
• Unleash LLMs Potential for Recommendation by Coordinating Twin-Tower Dynamic Semantic Token Generator [60.07198935747619]
  We propose Twin-Tower Dynamic Semantic Recommender (T TDS), the first generative RS which adopts dynamic semantic index paradigm. To be more specific, we for the first time contrive a dynamic knowledge fusion framework which integrates a twin-tower semantic token generator into the LLM-based recommender. The proposed T TDS recommender achieves an average improvement of 19.41% in Hit-Rate and 20.84% in NDCG metric, compared with the leading baseline methods.
  arXiv  Detail & Related papers  (2024-09-14T01:45:04Z)
• Learnable Item Tokenization for Generative Recommendation [113.80559032128065]
  We propose LETTER (a LEarnable Tokenizer for generaTivE Recommendation), which integrates hierarchical semantics, collaborative signals, and code assignment diversity.<n> LETTER incorporates Residual Quantized VAE for semantic regularization, a contrastive alignment loss for collaborative regularization, and a diversity loss to mitigate code assignment bias.
  arXiv  Detail & Related papers  (2024-05-12T15:49:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.