Rethinking Item Tokenization in Generative Recommenders: From Fixed Atoms to Semantic Subwords

• URL: http://arxiv.org/abs/2608.22734v1
• Date: Mon, 24 Aug 2026 02:42:51 GMT
• Title: Rethinking Item Tokenization in Generative Recommenders: From Fixed Atoms to Semantic Subwords
• Abstract summary: In generative recommender systems, items are typically tokenized into fixed-length semantic ID sequences for autoregressive next-item prediction.<n>For user-context modeling, this fine-grained representation triggers Intra-item Attention Overload.<n>We propose Semantic Subword Tokenization (SST), which represents historical items as variable-length semantic subwords.
• Score: 50.621722853036005
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In generative recommender systems, items are typically tokenized into fixed-length semantic ID sequences for autoregressive next-item prediction. However, for user-context modeling, this fine-grained representation triggers Intra-item Attention Overload: excessive attention is spent on low-level intra-item dependencies rather than high-level inter-item behavioral transitions. To address this, we propose Semantic Subword Tokenization (SST), which represents historical items as variable-length semantic subwords while preserving fixed-length target decoding. SST first applies Item-level Subword Tokenization (IST) to merge stable adjacent atom tokens into compact semantic subword tokens, thereby reducing intra-item reassembly in the encoder. It then introduces Behavior-induced Co-occurrence Augmentation (BCA) to inject coarse-grained semantic prefix transition signals, guiding the freed modeling capacity toward inter-item behavioral regularities. Extensive experiments on three public datasets and three generative recommender backbones show empirical improvements of SST over fixed-length and transferable variable-length SID baselines. Code is available at https://github.com/mxrcandy/Semantic-Subword-Tokenization.

Related papers

• Preference Shapes Relevance: Cross-component Hierarchical Semantic Alignment for Personalized Generative Retrieval [37.821022474615994]
  Generative Retrieval (GR) has emerged as a promising paradigm by mapping queries directly to Semantic IDs (SIDs) with powerful representation capabilities for candidate items.<n>Existing SIDs derived solely from item content create a semantic gap, failing to align dynamic query intents with static item representations.<n>We present a novel personalized GR framework that models user behavior by synergizing discrete SIDs for structural guidance and continuous representations for fine-grained semantic refinement.
  arXiv  Detail & Related papers  (2026-08-31T10:30:22Z)
• Learning from Unreachable Rewards: Hint-Conditioned Reinforcement Learning for Generative Recommendation [39.5807784132569]
  Semantic-ID generative recommenders represent each item as a short sequence of discrete semantic tokens and predict the next item by autoregressively generating this token sequence.<n>This paradigm enables a unified generation interface for item IDs, histories, and item text, but it also creates a structured optimization bottleneck during reward-based post-training.<n>We propose HCGRec, a semantic-ID generative recommendation framework that recovers learning signal for such hard training instances.<n>HCGRec substantially improves over supervised fine-tuning and vanilla reward-based post-training, while reducing zero-advantage training samples from over 70% to below 20%.
  arXiv  Detail & Related papers  (2026-08-12T12:13:08Z)
• TopoGR: Revealing and Preserving Latent Structure of Semantic ID in Generative Recommendation [25.345503799250476]
  TopoGR is a topology-preserving generative recommendation framework based on Bit-decomposable Semantic ID( Binary SID)<n>We show that TopoGR consistently outperforms existing state-of-the-art baselines in recommendation performance.
  arXiv  Detail & Related papers  (2026-07-28T02:45:33Z)
• CogRec: Structure-Cognitive Fast-and-Slow Reasoning for Generative Recommendation [29.663400191042996]
  CogRec is a structure-cognitive fast-and-slow reasoning framework.<n>CogRec augments the vertical SID hierarchy with intra-layer semantic graphs and item-level neighborhoods.<n> SID Routing represents recommendation reasoning through layer-wise Match, LateralJump, and Explore operations.
  arXiv  Detail & Related papers  (2026-07-27T13:19:55Z)
• PairAlign: A Framework for Sequence Tokenization via Self-Alignment with Applications to Audio Tokenization [8.126475706520063]
  PairAlign is a framework for compact audio tokenization through sequence-level self-alignment.<n>It learns compact, non-degenerate sequences with broad vocabulary usage.<n>It preserves edit-distance search while reducing archive token count by 55%.
  arXiv  Detail & Related papers  (2026-05-07T17:11:22Z)
• Multigrain-aware Semantic Prototype Scanning and Tri-Token Prompt Learning Embraced High-Order RWKV for Pan-Sharpening [46.94213480279033]
  We propose a Multigrain-aware Semantic Prototype Scanning paradigm for pan-sharpening.<n>We use a high-order RWKV architecture and a tri-token prompting mechanism derived from semantic clustering.<n> Experimental results demonstrate the superiority of our method.
  arXiv  Detail & Related papers  (2026-04-16T05:10:03Z)
• Trie-Aware Transformers for Generative Recommendation [18.901863061905825]
  We propose TrieRec, a trie-aware generative recommendation method that augments Transformers with structural inductive biases.<n>We implement TrieRec within three representative GR backbones, achieving notably improvements of 8.83% on average across four real-world datasets.
  arXiv  Detail & Related papers  (2026-02-25T08:25:16Z)
• C2T-ID: Converting Semantic Codebooks to Textual Document Identifiers for Generative Search [73.61009656398384]
  We propose C2T-ID, which builds semantic numerical docid via hierarchical clustering.<n>C2T-ID significantly outperforms atomic, semantic codebook, and pure-text docid baselines.
  arXiv  Detail & Related papers  (2025-10-22T04:05:38Z)
• Learning Multi-Aspect Item Palette: A Semantic Tokenization Framework for Generative Recommendation [55.99632509895994]
  We introduce LAMIA, a novel approach for multi-aspect semantic tokenization.<n>Unlike RQ-VAE, which uses a single embedding, LAMIA learns an item palette''--a collection of independent and semantically parallel embeddings.<n>Our results demonstrate significant improvements in recommendation accuracy over existing methods.
  arXiv  Detail & Related papers  (2024-09-11T13:49:48Z)
• CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens [49.569695524535454]
  We propose to represent speech with supervised semantic tokens, which are derived from a multilingual speech recognition model by inserting vector quantization into the encoder. Based on the tokens, we further propose a scalable zero-shot TTS synthesizer, CosyVoice, which consists of an LLM for text-to-token generation and a conditional flow matching model for token-to-speech synthesis.
  arXiv  Detail & Related papers  (2024-07-07T15:16:19Z)
• Learnable Item Tokenization for Generative Recommendation [113.80559032128065]
  We propose LETTER (a LEarnable Tokenizer for generaTivE Recommendation), which integrates hierarchical semantics, collaborative signals, and code assignment diversity.<n> LETTER incorporates Residual Quantized VAE for semantic regularization, a contrastive alignment loss for collaborative regularization, and a diversity loss to mitigate code assignment bias.
  arXiv  Detail & Related papers  (2024-05-12T15:49:38Z)
• Pushdown Layers: Encoding Recursive Structure in Transformer Language Models [86.75729087623259]
  Recursion is a prominent feature of human language, and fundamentally challenging for self-attention. This work introduces Pushdown Layers, a new self-attention layer. Transformers equipped with Pushdown Layers achieve dramatically better and 3-5x more sample-efficient syntactic generalization.
  arXiv  Detail & Related papers  (2023-10-29T17:27:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.