It Takes Two to Match: Co-Evolving Generative Retriever with Reinforcement Learning

• URL: http://arxiv.org/abs/2609.00638v1
• Date: Tue, 01 Sep 2026 03:17:02 GMT
• Title: It Takes Two to Match: Co-Evolving Generative Retriever with Reinforcement Learning
• Abstract summary: CoGR trains generators to construct retrieval representations on both query and item sides.<n>Each generator produces a compact set of keywords, which are matched directly through an inverted index.<n>CoGR achieves the best performance on both an internal APP Marketplace dataset and the public WANDS benchmark.
• Score: 3.106071998555287
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Retrieval is the first stage of modern search and advertising systems, selecting a candidate set from a large item universe for downstream ranking and auction. Recent work increasingly leverages LLMs to improve retrieval through query expansion, data synthesis, and retrieval-feedback training. However, the generative component is typically used for query-side augmentation, while final matching is still delegated to a downstream retriever. We introduce CoGR, a retrieval framework that instead trains LLMs to directly construct retrieval representations on both query and item sides. Each generator produces a compact set of keywords, which are matched directly through an inverted index, preserving compatibility with existing keyword-based retrieval infrastructure. CoGR uses a two-stage training pipeline. Supervised fine-tuning first establishes an aligned keyword space, after which co-evolving reinforcement learning alternately optimizes the query- and item-side generators with GRPO against the opposite side's frozen index. Both sides optimize the same query-to-item retrieval $F_1$ objective: the query side receives retrieval $F_1$ directly, while the item side receives a counterfactual marginal reward measuring the change in query-side $F_1$ caused by its generated keywords. Across 10 representative sparse, dense, and generative baselines, CoGR achieves the best performance on both an internal APP Marketplace dataset and the public WANDS benchmark, improving $F_1$ over the strongest baseline by $10.9\%$ and $36.1\%$, respectively. Further analysis shows stable co-evolution and increasingly aligned query--item keyword spaces over training.

Related papers

• ORDER: Task-Conditioned Routing for Retrieval-Augmented Generation [3.6028767235872965]
  In this paper, we introduce a query-conditioned RAG framework that jointly adapts indexing and retrieval to the incoming query.<n>We evaluate our framework on large-scale, heterogeneous historical archives and show that conditioning both indexing and retrieval on the query consistently outperforms both naive baselines and strong state-of-the-art RAG systems.
  arXiv  Detail & Related papers  (2026-09-15T11:23:16Z)
• TSGR: Taobao Search Generative Retrieval [32.4338892898415]
  We propose a unified generative retrieval framework that incorporates value awareness into both item representation and candidate ranking.<n>$textbfT$aobao $textbfS$earch $textbfG$enerative $textbfR$etrieval ($textbfTSGR$)<n>$textbfVRM$ that is built upon and jointly optimized with the GR, enabling a single model to seamlessly serve as both retriever and pre-ranker.
  arXiv  Detail & Related papers  (2026-07-21T07:17:43Z)
• GrepSeek: Training Search Agents for Direct Corpus Interaction [66.69568141699311]
  GrepSeek is an optimized direct corpus interaction (DCI) search agent that trains a compact search agent to find, filter, and compose evidence from large text corpora.<n>To make DCI practical at scale, we further use a semantics-preserving sharded-parallel execution engine that accelerates shell-based retrieval by up to $7.6times$ while preserving byte-exact equivalence with sequential execution of the shell command.
  arXiv  Detail & Related papers  (2026-05-28T03:37:33Z)
• Subtraction Gets You More: Gap-Aware Retrieval for Multimodal Multi-Hop QA [11.316299961548415]
  We propose a paradigm that performs implicit query rewriting directly at the embedding level.<n>By context-subtractive query steering, GRAIL excels at compositional cross-modal reasoning.<n>By dynamically routing queries based on task type, our Hybrid Framework achieves a 40.3% macro-averaged performance gain on MultimodalQA.
  arXiv  Detail & Related papers  (2026-05-27T15:46:21Z)
• CoSearch: Joint Training of Reasoning and Document Ranking via Reinforcement Learning for Agentic Search [51.911048955965136]
  CoSearch is a framework that jointly trains a multi-step reasoning agent and a generative document ranking model.<n>Our results show that joint training of the reasoning agent and retrieval system is both feasible and strongly performant.
  arXiv  Detail & Related papers  (2026-04-19T17:48:17Z)
• AlignCoder: Aligning Retrieval with Target Intent for Repository-Level Code Completion [55.21541958868449]
  We propose AlignCoder, a repository-level code completion framework.<n>Our framework generates an enhanced query that bridges the semantic gap between the initial query and the target code.<n>We employ reinforcement learning to train an AlignRetriever that learns to leverage inference information in the enhanced query for more accurate retrieval.
  arXiv  Detail & Related papers  (2026-01-27T15:23:14Z)
• Rethinking On-policy Optimization for Query Augmentation [49.87723664806526]
  We present the first systematic comparison of prompting-based and RL-based query augmentation across diverse benchmarks.<n>We introduce a novel hybrid method, On-policy Pseudo-document Query Expansion (OPQE), which learns to generate a pseudo-document that maximizes retrieval performance.
  arXiv  Detail & Related papers  (2025-10-20T04:16:28Z)
• Can Query Expansion Improve Generalization of Strong Cross-Encoder Rankers? [72.42500059688396]
  We show that it is possible to improve the generalization of a strong neural ranker, by prompt engineering and aggregating the ranking results of each expanded query via fusion. Experiments on BEIR and TREC Deep Learning show that the nDCG@10 scores of both MonoT5 and RankT5 following these steps are improved.
  arXiv  Detail & Related papers  (2023-11-15T18:11:41Z)
• UnifieR: A Unified Retriever for Large-Scale Retrieval [84.61239936314597]
  Large-scale retrieval is to recall relevant documents from a huge collection given a query. Recent retrieval methods based on pre-trained language models (PLM) can be coarsely categorized into either dense-vector or lexicon-based paradigms. We propose a new learning framework, UnifieR which unifies dense-vector and lexicon-based retrieval in one model with a dual-representing capability.
  arXiv  Detail & Related papers  (2022-05-23T11:01:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.