Efficient Generative Retrieval for E-commerce Search with Semantic Cluster IDs and Expert-Guided RL

• URL: http://arxiv.org/abs/2605.14434v1
• Date: Thu, 14 May 2026 06:27:46 GMT
• Title: Efficient Generative Retrieval for E-commerce Search with Semantic Cluster IDs and Expert-Guided RL
• Abstract summary: Generative retrieval offers a promising alternative by unifying the fragmented multi-stage retrieval process into a single end-to-end model.<n>We propose a retrieval framework tailored for real-world recall scenarios, positioning generative retrieval as a recall-stage supplement rather than an end-to-end replacement.
• Score: 8.046510111008638
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative retrieval offers a promising alternative by unifying the fragmented multi-stage retrieval process into a single end-to-end model. However, its practical adoption in industrial e-commerce search remains challenging, given the massive and dynamic product catalogs, strict latency requirements, and the need to align retrieval with downstream ranking goals. In this work, we propose a retrieval framework tailored for real-world recall scenarios, positioning generative retrieval as a recall-stage supplement rather than an end-to-end replacement. Our method, CQ-SID (Category-and-Query constrained Semantic ID), employs category-aware and query-item contrastive learning along with Residual Quantized VAEs to encode items into hierarchical semantic cluster identifiers, significantly reducing beam search complexity. Additionally, we develop EG-GRPO (Expert-Guided Group Relative Policy Optimization), a reinforcement learning approach that aligns generative recall with downstream ranking under sparse rewards by injecting ground-truth samples to stabilize training. Offline experiments on TmallAPP search logs show that CQ-SID achieves up to 26.76% and 11.11% relative gains in semantic and personalized click hitrate over RQ-VAE baselines, while halving beam search size. EG-GRPO further improves multi-objective performance. Online A/B tests confirm gains in GMV (+1.15%) and UCTCVR (+0.40%). The generative recall channel now contributes substantially in production, accounting for over 50.25% of exposures, 58.96% of clicks, and 72.63% of purchases, demonstrating a viable path for deploying generative retrieval in real-world e-commerce systems.

Related papers

• VARG: Value-Aware and Ranking-Aligned Generative Retrieval for Dynamic E-commerce Search [7.64283018838172]
  VARG is a generative retrieval system for Tmall App search that directly admits generated item candidates to the existing final ranker.<n>Three-stage supervised fine-tuning progressively learns item-to-identifier mappings, query-semantic retrieval, and personalized retrieval.<n>Prefix-GRPO combines gated rewards based on output legality, user behavior, ranker advantage, and search relevance with prefix-aware token weighting to align candidate generation with business value and ranking objectives.
  arXiv  Detail & Related papers  (2026-09-13T12:59:04Z)
• TSGR: Taobao Search Generative Retrieval [32.4338892898415]
  We propose a unified generative retrieval framework that incorporates value awareness into both item representation and candidate ranking.<n>$textbfT$aobao $textbfS$earch $textbfG$enerative $textbfR$etrieval ($textbfTSGR$)<n>$textbfVRM$ that is built upon and jointly optimized with the GR, enabling a single model to seamlessly serve as both retriever and pre-ranker.
  arXiv  Detail & Related papers  (2026-07-21T07:17:43Z)
• CoSearch: Joint Training of Reasoning and Document Ranking via Reinforcement Learning for Agentic Search [51.911048955965136]
  CoSearch is a framework that jointly trains a multi-step reasoning agent and a generative document ranking model.<n>Our results show that joint training of the reasoning agent and retrieval system is both feasible and strongly performant.
  arXiv  Detail & Related papers  (2026-04-19T17:48:17Z)
• GenFacet: End-to-End Generative Faceted Search via Multi-Task Preference Alignment in E-Commerce [12.502808629463702]
  We introduce GenFacet, an end-to-end generative framework deployed at JD.com.<n>GenFacet reframes faceted search as two coupled generative tasks within a unified Large Language Model.
  arXiv  Detail & Related papers  (2026-03-20T06:01:01Z)
• Test-Time Strategies for More Efficient and Accurate Agentic RAG [58.44913384057518]
  Retrieval-Augmented Generation (RAG) systems face challenges with complex, multihop questions.<n>Such approaches can introduce inefficiencies, including repetitive retrieval of previously processed information.<n>We investigate test-time modifications to the Search-R1 pipeline to mitigate these identified shortcomings.
  arXiv  Detail & Related papers  (2026-03-12T19:18:59Z)
• Aligning Large Language Models with Searcher Preferences [26.974618053554394]
  We introduce SearchLLM, the first large language model (LLM) for open-ended generative search.<n>We design a hierarchical, multi-dimensional reward system that separates bottom-line constraints.<n> offline evaluations and online A/B tests show improved generation quality and user engagement.
  arXiv  Detail & Related papers  (2026-03-11T06:44:30Z)
• Beyond Relevance: On the Relationship Between Retrieval and RAG Information Coverage [89.58253972744531]
  Retrieval-augmented generation (RAG) systems combine document retrieval with a generative model to address complex information seeking tasks like report generation.<n>We investigate whether upstream retrieval metrics can serve as reliable early indicators of the final generated response's information coverage.<n>Our findings demonstrate strong correlations between coverage-based retrieval metrics and nugget coverage in generated responses at both topic and system levels.
  arXiv  Detail & Related papers  (2026-03-09T18:20:20Z)
• FORGE: Forming Semantic Identifiers for Generative Retrieval in Industrial Datasets [64.51403245281547]
  FORGE is a benchmark for FOrming semantic identifieR in Generative rEtrieval with industrial datasets.<n>For real-world applications, FORGE introduces an offline pretraining schema that reduces online convergence by half.
  arXiv  Detail & Related papers  (2025-09-25T08:44:22Z)
• OneSearch: A Preliminary Exploration of the Unified End-to-End Generative Framework for E-commerce Search [43.94443394870866]
  OneSearch is the first industrial-deployed end-to-end generative framework for e-commerce search.<n>OneSearch reduces operational expenditure by 75.40% and improves Model FLOPs Utilization from 3.26% to 27.32%.<n>The system has been successfully deployed across multiple search scenarios in Kuaishou, serving millions of users.
  arXiv  Detail & Related papers  (2025-09-03T11:50:04Z)
• MMSearch-R1: Incentivizing LMMs to Search [49.889749277236376]
  We present MMSearch-R1, the first end-to-end reinforcement learning framework that enables on-demand, multi-turn search in real-world Internet environments.<n>Our framework integrates both image and text search tools, allowing the model to reason about when and how to invoke them guided by an outcome-based reward with a search penalty.
  arXiv  Detail & Related papers  (2025-06-25T17:59:42Z)
• Adaptive Retrieval and Scalable Indexing for k-NN Search with Cross-Encoders [77.84801537608651]
  Cross-encoder (CE) models which compute similarity by jointly encoding a query-item pair perform better than embedding-based models (dual-encoders) at estimating query-item relevance. We propose a sparse-matrix factorization based method that efficiently computes latent query and item embeddings to approximate CE scores and performs k-NN search with the approximate CE similarity.
  arXiv  Detail & Related papers  (2024-05-06T17:14:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.