Discrimination Is Generation: Unifying Ranking and Retrieval from a Tokenizer Perspective

• URL: http://arxiv.org/abs/2605.14853v1
• Date: Thu, 14 May 2026 13:59:29 GMT
• Title: Discrimination Is Generation: Unifying Ranking and Retrieval from a Tokenizer Perspective
• Abstract summary: We propose DIG (textbfDiscrimination textbfIs textbfGeneration), which embeds the tokenizer inside a discriminative ranking model for end-to-end training.<n> Experiments on three public benchmarks and two industrial datasets demonstrate that DIG simultaneously improves ranking, retrieval, and unified retrieval-ranking quality.
• Score: 14.332200648147863
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semantic IDs (SIDs) define the generation space of generative recommendation and directly determine its personalization ceiling. However, existing tokenizers are trained independently with retrieval objectives, leaving personalization signals fully decoupled from the SID construction process -- a fundamental gap that causes generative retrieval to persistently lag behind discriminative ranking. In this paper, we rethink the essence of SIDs: \emph{ranking seeks argmax in item space while retrieval seeks argmax in token space; both are the same problem solved at different granularities.} Based on this insight, we propose \DIG (\textbf{D}iscrimination \textbf{I}s \textbf{G}eneration), which embeds the tokenizer inside a discriminative ranking model for end-to-end training -- the ranker naturally becomes a retrieval model, yielding two models from a single training run. \DIG is organized around a \emph{feature assignment taxonomy}: item-intrinsic static features are encoded into SIDs, user-item cross features (u2i) implicitly drive codebook boundaries toward recommendation decision boundaries during training, and an MLP$_\mathrm{u2t}$ distillation module approximates u2i at the token level for inference. Experiments on three public benchmarks and two industrial datasets demonstrate that \DIG simultaneously improves ranking, retrieval, and unified retrieval-ranking quality.

Related papers

• Preference Shapes Relevance: Cross-component Hierarchical Semantic Alignment for Personalized Generative Retrieval [37.821022474615994]
  Generative Retrieval (GR) has emerged as a promising paradigm by mapping queries directly to Semantic IDs (SIDs) with powerful representation capabilities for candidate items.<n>Existing SIDs derived solely from item content create a semantic gap, failing to align dynamic query intents with static item representations.<n>We present a novel personalized GR framework that models user behavior by synergizing discrete SIDs for structural guidance and continuous representations for fine-grained semantic refinement.
  arXiv  Detail & Related papers  (2026-08-31T10:30:22Z)
• Interpretable Representation via LLM-Driven Generative Disentanglement for Local-Life Service Recommendation [28.83516611687355]
  We propose Interpretable Representation via LLM-Driven Generative Disentanglement for Local-Life Service Recommendation (LGRID)<n>LGRID introduces a generative disentanglement paradigm through an Encode -> Disentangle -> Align -> Quantize pipeline.<n> Experiments on Kuaishou and Foursquare show that LGRID consistently outperforms strong SID baselines.
  arXiv  Detail & Related papers  (2026-07-30T09:52:18Z)
• TopoGR: Revealing and Preserving Latent Structure of Semantic ID in Generative Recommendation [25.345503799250476]
  TopoGR is a topology-preserving generative recommendation framework based on Bit-decomposable Semantic ID( Binary SID)<n>We show that TopoGR consistently outperforms existing state-of-the-art baselines in recommendation performance.
  arXiv  Detail & Related papers  (2026-07-28T02:45:33Z)
• Guess Where You Go: Generative Next Point-of-Interest Recommendation in Amap [12.259216278301743]
  We propose an end-to-end industrial framework that integrates semantic identifier (SID) generation with generative Point-of-Interest (POI) recommendation.<n>Gwhere is deployed in Amap's homepage service under high-concurrency and low-latency constraints.<n>Long-term online A/B tests show improvements of 5.83% in P-CTR and 6.20% in U-CTR over the production baseline.
  arXiv  Detail & Related papers  (2026-07-13T15:44:06Z)
• From Local Indices to Global Identifiers: Generative Reranking for Recommender Systems via Global Action Space [50.72071213515985]
  GloRank is a generative framework that shifts reranking from selecting local indices to generating global identifiers.<n>We show that GloRank consistently outperforms state-of-the-art baselines and achieves superior robustness in cold-start scenarios.
  arXiv  Detail & Related papers  (2026-04-28T06:57:00Z)
• UniRec: Bridging the Expressive Gap between Generative and Discriminative Recommendation via Chain-of-Attribute [12.89120699793625]
  Generative Recommendation (GR) reframes retrieval and ranking as autoregressive decoding over Semantic IDs (SIDs)<n>We formalize this via Bayes' theorem: ranking by p(y|f,u) is equivalent to ranking by p(f|y,u), which factorizes autoregressively over item features.<n>We propose UniRec with Chain-of-Attribute (CoA) as its core mechanism. CoA prefixes each SID sequence with structured attribute tokens:category, seller, brand, before decoding the SID, recovering the item-side feature crossing that discriminative models exploit.
  arXiv  Detail & Related papers  (2026-04-14T03:13:50Z)
• End-to-End Semantic ID Generation for Generative Advertisement Recommendation [33.453121305193434]
  We propose a Unified SID generation framework for generative advertisement recommendation.<n>Specifically, we jointly optimize embeddings and SIDs in an end-to-end manner from raw advertising data.<n>Experiments demonstrate that UniSID consistently outperforms state-of-the-art SID generation methods.
  arXiv  Detail & Related papers  (2026-02-11T02:38:26Z)
• Differentiable Semantic ID for Generative Recommendation [65.83703273297492]
  Generative recommendation provides a novel paradigm in which each item is represented by a discrete semantic ID (SID) learned from rich content.<n>In practice, SIDs are typically optimized only for content reconstruction rather than recommendation accuracy.<n>A natural approach is to make semantic indexing differentiable so that recommendation gradients can directly influence SID learning.<n>We propose DIGER, a first step toward effective differentiable semantic IDs for generative recommendation.
  arXiv  Detail & Related papers  (2026-01-27T15:34:11Z)
• DiffuGR: Generative Document Retrieval with Diffusion Language Models [80.78126312115087]
  We propose generative document retrieval with diffusion language models, dubbed DiffuGR.<n>For inference, DiffuGR attempts to generate DocID tokens in parallel and refine them through a controllable number of denoising steps.<n>In contrast to conventional left-to-right auto-regressive decoding, DiffuGR provides a novel mechanism to first generate more confident DocID tokens.
  arXiv  Detail & Related papers  (2025-11-11T12:00:09Z)
• DiffGRM: Diffusion-based Generative Recommendation Model [63.35379395455103]
  Generative recommendation (GR) is an emerging paradigm that represents each item via a tokenizer as an n-digit semantic ID (SID)<n>We propose DiffGRM, a diffusion-based GR model that replaces the autoregressive decoder with a masked discrete diffusion model (MDM)<n> Experiments show consistent gains over strong generative and discriminative recommendation baselines on multiple datasets.
  arXiv  Detail & Related papers  (2025-10-21T03:23:32Z)
• Language Models As Semantic Indexers [78.83425357657026]
  We introduce LMIndexer, a self-supervised framework to learn semantic IDs with a generative language model. We show the high quality of the learned IDs and demonstrate their effectiveness on three tasks including recommendation, product search, and document retrieval.
  arXiv  Detail & Related papers  (2023-10-11T18:56:15Z)
• Dual-Refinement: Joint Label and Feature Refinement for Unsupervised Domain Adaptive Person Re-Identification [51.98150752331922]
  Unsupervised domain adaptive (UDA) person re-identification (re-ID) is a challenging task due to the missing of labels for the target domain data. We propose a novel approach, called Dual-Refinement, that jointly refines pseudo labels at the off-line clustering phase and features at the on-line training phase. Our method outperforms the state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2020-12-26T07:35:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.