TokenMinds: Pretrained User Tokens and Embeddings for User Understanding in Large Recommender Systems

• URL: http://arxiv.org/abs/2606.25147v1
• Date: Tue, 23 Jun 2026 20:20:37 GMT
• Title: TokenMinds: Pretrained User Tokens and Embeddings for User Understanding in Large Recommender Systems
• Abstract summary: TokenMinds is an industrial-scale system that extends the PLUM framework from item retrieval to user modeling.<n>It generates both discrete SID-based user tokens and dense user embeddings via an encoder-decoder architecture adapted from pre-trained LLMs.
• Score: 22.086361560657593
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: User modeling in industrial recommender systems typically produces dense embeddings, which suffer from representational constraints inherent to fixed-dimensional vectors. An emerging alternative for discrete user representation -- using LLMs to generate text-based user tokens -- captures topical co-occurrences rather than deep sequential behavior dynamics and produces outputs that are difficult to ground to item attributes. Meanwhile, Semantic ID (SID) based item tokenization has proven effective for improving generalization in generative recommendation, yet discrete SID-based representations for users remain largely unexplored. We propose TokenMinds, an industrial-scale system that extends the PLUM framework from item retrieval to user modeling, generating both discrete SID-based user tokens and dense user embeddings via an encoder-decoder architecture adapted from pre-trained LLMs. This dual-output design provides the complementary benefits of discrete, semantically grounded user representations while maintaining compatibility with existing downstream models that rely on dense embeddings. Additionally, the shared SID vocabulary naturally extends to cross-scenario modeling: by unifying long-form and short-form video behaviors into a single model, we substantially reduce training and serving costs. We validate TokenMinds through extensive offline experiments and live launches on multiple YouTube surfaces, served on full user traffic (billions of users) via an asynchronous infrastructure that decouples representation generation from downstream scoring. Focusing on ranking as the primary downstream use case, our results confirm the practical viability of SID-based user tokens at industrial scale and demonstrate that tokens and dense embeddings provide complementary value across different production ranking systems.

Related papers

• Shape Your Feed: An LLM-based Agentic System for Conversational Recommendation [18.690831845179797]
  Shape Your Feed (SYF) is an agentic recommendation framework that enables real-time, multimodal co-curation of content.<n>SYF employs a three-tier architecture: (i) a Perception Flow that captures user intent from text prompts, voice commands, and UI interactions; (ii) a Serving Flow that performs real-time agentic re-ranking and pruning of candidate items; and (iii) a Self-Evolution Flow that aligns system behavior with human judgments.
  arXiv  Detail & Related papers  (2026-08-06T22:54:16Z)
• Tokens are All You Need: Dual-purpose Semantic IDs for Achieving LLM-Level I/O Efficiency in recommendation systems [11.795172547249]
  Large-scale recommendation systems face "Memory Wall" bottlenecks due to massive, dense embedding tables.<n>Inspired by computer vision data compression, we propose Dual-purpose Semantic IDs to achieve LLM-level I/O efficiency.
  arXiv  Detail & Related papers  (2026-07-26T18:04:19Z)
• SlimPer: Make Personalization Model Slim and Smart [20.68857468213109]
  We propose SlimPer to reformulate personalized ranking as iterative refinement of a compact, unified user, item> knowledge base.<n>SlimPer unifies sparse, dense, and sequence features within a single backbone and provides inherent interpretability through its attention mechanism.
  arXiv  Detail & Related papers  (2026-07-14T02:31:32Z)
• From Local Indices to Global Identifiers: Generative Reranking for Recommender Systems via Global Action Space [50.72071213515985]
  GloRank is a generative framework that shifts reranking from selecting local indices to generating global identifiers.<n>We show that GloRank consistently outperforms state-of-the-art baselines and achieves superior robustness in cold-start scenarios.
  arXiv  Detail & Related papers  (2026-04-28T06:57:00Z)
• Query as Anchor: Scenario-Adaptive User Representation via Large Language Model [28.30329175937291]
  We propose Query-as-Anchor, a framework shifting user modeling from static encoding to dynamic, query-aware synthesis.<n>We first construct UserU, an industrial-scale pre-training dataset that aligns behavioral sequences with user understanding semantics.<n>We introduce Cluster-based Soft Prompt Tuning to enforce discriminative latent structures.<n>For deployment, anchoring queries at sequence termini enables KV-cache-accelerated inference with negligible incremental latency.
  arXiv  Detail & Related papers  (2026-02-16T06:09:31Z)
• GLASS: A Generative Recommender for Long-sequence Modeling via SID-Tier and Semantic Search [51.44490997013772]
  GLASS is a novel framework that integrates long-term user interests into the generative process via SID-Tier and Semantic Search.<n>We show that GLASS outperforms state-of-the-art baselines in experiments on two large-scale real-world datasets.
  arXiv  Detail & Related papers  (2026-02-05T13:48:33Z)
• Masked Diffusion for Generative Recommendation [30.8737219110446]
  Generative recommendation (GR) with semantic IDs (SIDs) has emerged as a promising alternative to traditional recommendation approaches.<n>We propose to instead model and learn the probability of a user's sequence of SIDs using masked diffusion.<n>We demonstrate through thorough experiments that our proposed method consistently outperforms autoregressive modeling.
  arXiv  Detail & Related papers  (2025-11-28T09:36:26Z)
• Multi-agents based User Values Mining for Recommendation [52.26100802380767]
  We propose a zero-shot multi-LLM collaborative framework for effective and accurate user value extraction.<n>We apply text summarization techniques to condense item content while preserving essential meaning.<n>To mitigate hallucinations, we introduce two specialized agent roles: evaluators and supervisors.
  arXiv  Detail & Related papers  (2025-05-02T04:01:31Z)
• Diffusion Generative Recommendation with Continuous Tokens [21.222713476105195]
  ContRec is a framework that seamlessly integrates continuous tokens into LLM-based RecSys.<n>We show that ContRec consistently outperforms both traditional and SOTA LLM-based recommender systems.<n>Our results highlight the potential of continuous tokenization and generative modeling for advancing the next generation of recommender systems.
  arXiv  Detail & Related papers  (2025-04-16T12:01:03Z)
• Learning Multi-Aspect Item Palette: A Semantic Tokenization Framework for Generative Recommendation [55.99632509895994]
  We introduce LAMIA, a novel approach for multi-aspect semantic tokenization.<n>Unlike RQ-VAE, which uses a single embedding, LAMIA learns an item palette''--a collection of independent and semantically parallel embeddings.<n>Our results demonstrate significant improvements in recommendation accuracy over existing methods.
  arXiv  Detail & Related papers  (2024-09-11T13:49:48Z)
• MMGRec: Multimodal Generative Recommendation with Transformer Model [81.61896141495144]
  MMGRec aims to introduce a generative paradigm into multimodal recommendation. We first devise a hierarchical quantization method Graph CF-RQVAE to assign Rec-ID for each item from its multimodal information. We then train a Transformer-based recommender to generate the Rec-IDs of user-preferred items based on historical interaction sequences.
  arXiv  Detail & Related papers  (2024-04-25T12:11:27Z)
• MISSRec: Pre-training and Transferring Multi-modal Interest-aware Sequence Representation for Recommendation [61.45986275328629]
  We propose MISSRec, a multi-modal pre-training and transfer learning framework for sequential recommendation. On the user side, we design a Transformer-based encoder-decoder model, where the contextual encoder learns to capture the sequence-level multi-modal user interests. On the candidate item side, we adopt a dynamic fusion module to produce user-adaptive item representation.
  arXiv  Detail & Related papers  (2023-08-22T04:06:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.