How Reliable Are Semantic-ID Tokenizer Comparisons in Generative Recommendation?

• URL: http://arxiv.org/abs/2605.25330v1
• Date: Mon, 25 May 2026 01:22:45 GMT
• Title: How Reliable Are Semantic-ID Tokenizer Comparisons in Generative Recommendation?
• Abstract summary: In generative recommendation, each item is represented as a sequence of discrete codes, and an autoregressive model is trained to generate the SID sequence of the next item.<n>This evaluation protocol equates SID-level matching with item-level recommendation, an equivalence that holds only when every SID sequence maps to a single item.<n>We show this assumption breaks down in practice: because tokenizers compress item features into a code space, semantically similar but collaboratively distinct items are frequently assigned the same SID sequence.
• Score: 3.9133310867939177
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In Semantic-ID (SID) based generative recommendation, each item is represented as a sequence of discrete codes, and an autoregressive model is trained to generate the SID sequence of the next item; top-K performance is then measured by checking whether the SID sequence of the target item appears among the generated sequences. This evaluation protocol equates SID-level matching with item-level recommendation, an equivalence that holds only when every SID sequence maps to a single item. We show this assumption breaks down in practice: because tokenizers compress item features into a code space, semantically similar but collaboratively distinct items are frequently assigned the same SID sequence. Across four datasets and five representative tokenizers, the fraction of items involved in such collisions reaches 30.5%, so matching a shared SID sequence identifies only a collision group rather than the target item. Consequently, SID-level metrics overestimate item-level performance (Hit@10 is inflated by up to 103.36%), and the inflation grows with the collision rate. To support faithful comparison, we develop collision-aware item-level metrics computed directly from generated SID sequences, together with a post-tokenizer procedure that reassigns last-level SIDs at minimum cost to obtain a collision-free assignment for any existing tokenizer. Our results indicate that SID-level rankings in prior work should be interpreted with caution, and that reliable tokenizer evaluation requires either item-level correction or collision-free SID assignments.

Related papers

• Rethinking Item Tokenization in Generative Recommenders: From Fixed Atoms to Semantic Subwords [50.621722853036005]
  In generative recommender systems, items are typically tokenized into fixed-length semantic ID sequences for autoregressive next-item prediction.<n>For user-context modeling, this fine-grained representation triggers Intra-item Attention Overload.<n>We propose Semantic Subword Tokenization (SST), which represents historical items as variable-length semantic subwords.
  arXiv  Detail & Related papers  (2026-08-24T02:42:51Z)
• TopoGR: Revealing and Preserving Latent Structure of Semantic ID in Generative Recommendation [25.345503799250476]
  TopoGR is a topology-preserving generative recommendation framework based on Bit-decomposable Semantic ID( Binary SID)<n>We show that TopoGR consistently outperforms existing state-of-the-art baselines in recommendation performance.
  arXiv  Detail & Related papers  (2026-07-28T02:45:33Z)
• Understanding Semantic IDs: From Item Representation to Item Selection in Generative Recommendation [7.786317729444243]
  We systematically investigate SIDs from item encoding and SID construction to autoregressive generation and final recommendation.<n>We show that SIDs retain broad organization but lose much of the encoder's fine local structure, while their exact tokens are not determined by item meaning alone.<n>We propose Item-Supported Decoding (ISD), a lightweight inference-time method that allows a user-specific item ranking to support corresponding SID prefixes before beam search discards them.
  arXiv  Detail & Related papers  (2026-07-27T18:52:39Z)
• Gryphon: A Unified Architecture for Semantic-ID Generation and Item-Level Scoring in Industrial Recommendations [0.0]
  We introduce a jointly trained item-level scoring component alongside SID generation.<n>Instead of ranking SIDs by accumulated token likelihood, Gryphon resolves each generated SID to its concrete items.<n>On an industrial music service, with item-level scoring trained under a next-item-prediction objective, Gryphon attains the highest item-level Recall@1000.
  arXiv  Detail & Related papers  (2026-06-07T12:31:26Z)
• SAPO: Step-Aligned Policy Optimization for Reasoning-Based Generative Recommendation [43.98709365191658]
  Generative recommendation treats next-item prediction as autoregressive item-identifier generation.<n>Recent work augments this paradigm with reasoning traces and optimize them via reinforcement learning with verifiable rewards.<n>In large-catalog recommendation, exact-match feedback on the generated SID only reports whether the final item is correct.
  arXiv  Detail & Related papers  (2026-05-17T20:53:08Z)
• Conditional Memory Enhanced Item Representation for Generative Recommendation [46.59623438612085]
  Generative recommendation (GR) has emerged as a promising paradigm that predicts target items by autoregressively generating their semantic identifiers (SID)<n>Most GR methods follow a quantization-representation-generation pipeline, first assigning each item a SID, then constructing input representations from SID-token embeddings, and finally predicting the target SID through autoregressive generation.<n>We propose ComeIR, a Conditional Memory enhanced Item Representation framework that reconstructs SID-token embeddings into item-aware inputs and restores the token granularity during SID decoding.
  arXiv  Detail & Related papers  (2026-05-12T02:56:59Z)
• UniRec: Bridging the Expressive Gap between Generative and Discriminative Recommendation via Chain-of-Attribute [12.89120699793625]
  Generative Recommendation (GR) reframes retrieval and ranking as autoregressive decoding over Semantic IDs (SIDs)<n>We formalize this via Bayes' theorem: ranking by p(y|f,u) is equivalent to ranking by p(f|y,u), which factorizes autoregressively over item features.<n>We propose UniRec with Chain-of-Attribute (CoA) as its core mechanism. CoA prefixes each SID sequence with structured attribute tokens:category, seller, brand, before decoding the SID, recovering the item-side feature crossing that discriminative models exploit.
  arXiv  Detail & Related papers  (2026-04-14T03:13:50Z)
• End-to-End Semantic ID Generation for Generative Advertisement Recommendation [33.453121305193434]
  We propose a Unified SID generation framework for generative advertisement recommendation.<n>Specifically, we jointly optimize embeddings and SIDs in an end-to-end manner from raw advertising data.<n>Experiments demonstrate that UniSID consistently outperforms state-of-the-art SID generation methods.
  arXiv  Detail & Related papers  (2026-02-11T02:38:26Z)
• Differentiable Semantic ID for Generative Recommendation [65.83703273297492]
  Generative recommendation provides a novel paradigm in which each item is represented by a discrete semantic ID (SID) learned from rich content.<n>In practice, SIDs are typically optimized only for content reconstruction rather than recommendation accuracy.<n>A natural approach is to make semantic indexing differentiable so that recommendation gradients can directly influence SID learning.<n>We propose DIGER, a first step toward effective differentiable semantic IDs for generative recommendation.
  arXiv  Detail & Related papers  (2026-01-27T15:34:11Z)
• DiffGRM: Diffusion-based Generative Recommendation Model [63.35379395455103]
  Generative recommendation (GR) is an emerging paradigm that represents each item via a tokenizer as an n-digit semantic ID (SID)<n>We propose DiffGRM, a diffusion-based GR model that replaces the autoregressive decoder with a masked discrete diffusion model (MDM)<n> Experiments show consistent gains over strong generative and discriminative recommendation baselines on multiple datasets.
  arXiv  Detail & Related papers  (2025-10-21T03:23:32Z)
• Order-agnostic Identifier for Large Language Model-based Generative Recommendation [94.37662915542603]
  Items are assigned identifiers for Large Language Models (LLMs) to encode user history and generate the next item.<n>Existing approaches leverage either token-sequence identifiers, representing items as discrete token sequences, or single-token identifiers, using ID or semantic embeddings.<n>We propose SETRec, which leverages semantic tokenizers to obtain order-agnostic multi-dimensional tokens.
  arXiv  Detail & Related papers  (2025-02-15T15:25:38Z)
• CycAs: Self-supervised Cycle Association for Learning Re-identifiable Descriptions [61.724894233252414]
  This paper proposes a self-supervised learning method for the person re-identification (re-ID) problem. Existing unsupervised methods usually rely on pseudo labels, such as those from video tracklets or clustering. We introduce a different unsupervised method that allows us to learn pedestrian embeddings from raw videos, without resorting to pseudo labels.
  arXiv  Detail & Related papers  (2020-07-15T09:52:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.