The Case Against Generation for Retrieval: Discriminative Language Models as Effective Retrievers

• URL: http://arxiv.org/abs/2607.25346v2
• Date: Sat, 01 Aug 2026 06:32:07 GMT
• Title: The Case Against Generation for Retrieval: Discriminative Language Models as Effective Retrievers
• Abstract summary: Large Language Models (LLMs) have emerged as powerful assets for recommender systems.<n> deploying them as generative recommenders or zero-shot rankers at web-scale remains bottlenecked by prohibitive computational overhead and grounding challenges.<n>In this paper, we revitalize the classic, highly efficient two-tower retrieval architecture by adapting LLMs as semantic representation backbones rather than generative engines.
• Score: 49.488975839992044
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) have emerged as powerful assets for recommender systems. However, deploying them as generative recommenders or zero-shot rankers at web-scale remains bottlenecked by prohibitive computational overhead and grounding challenges. In this paper, we revitalize the classic, highly efficient two-tower retrieval architecture by adapting LLMs as semantic representation backbones rather than generative engines. We introduce an LLM-native two-tower framework engineered for high-throughput, large-scale retrieval. Our architecture introduces several key innovations: a shared LLM encoder for joint user-item modeling, End-Of-Sentence (EOS) token pooling for compact sequence embedding, cross-dataset transfer learning, knowledge distillation from powerful cross-encoder teachers, and latent reasoning within the user tower. Extensive evaluation across three public benchmarks demonstrates that cross-encoder architecture outperforms current state-of-the-art (SoTA) models, while the efficient two-tower student achieves SoTA-comparable retrieval performance. Furthermore, experiments on internal large-scale production systems yield substantial topline retrieval improvements along with high resilience to model staleness and superior data scaling. Our findings demonstrate that when augmented with modern representation learning, the traditional two-tower paradigm remains an exceptionally competitive and practical solution for industrial retrieval systems.

Related papers

• ROSE: Retrieval-Oriented Segmentation Enhancement [79.7370065070812]
  Existing segmentation models based on multimodal large language models (MLLMs) often struggle with novel or emerging entities due to their inability to incorporate up-to-date knowledge.<n>We introduce the Novel Emerging Task (NEST), which focuses on segmenting (i) novel entities that MLLMs fail to recognize due to their absence from training data, and (ii) emerging entities that exist within the model's knowledge but demand up-to-date information for accurate recognition.<n>Rose: Retrieval-Oriented Enhancement, a plug-and-play framework designed to augment any MLLM-based segmentation model,
  arXiv  Detail & Related papers  (2026-04-15T17:59:35Z)
• Efficient Retrieval Scaling with Hierarchical Indexing for Large Scale Recommendation [18.266884490896228]
  This paper explores whether it is possible to learn a hierarchical organization over the memory of foundational retrieval models.<n>We propose jointly learning a hierarchical index using cross-attention and residual quantization for large-scale retrieval models.<n>We present our real-world deployment at Meta, supporting daily advertisement recommendations for billions of Facebook and Instagram users.
  arXiv  Detail & Related papers  (2026-04-14T16:59:03Z)
• A Simple and Effective Framework for Symmetric Consistent Indexing in Large-Scale Dense Retrieval [11.72564658353791]
  Dense retrieval has become the industry standard in large-scale information retrieval systems due to its high efficiency and competitive accuracy.<n>The widely adopted dual-tower encoding architecture introduces inherent challenges, primarily representational space misalignment and retrieval index inconsistency.<n>This paper proposes a simple and effective framework named SCI comprising two synergistic modules.<n>We provide theoretical guarantees for our approach, with its effectiveness validated by results across public datasets and real-world e-commerce datasets.
  arXiv  Detail & Related papers  (2025-12-15T08:11:24Z)
• TaoSearchEmb: A Multi-Objective Reinforcement Learning Framework for Dense Retrieval in Taobao Search [11.893855231479717]
  Retrieval-GRPO is a reinforcement learning-based dense retrieval framework.<n>It has been deployed on China's largest e-commerce platform.
  arXiv  Detail & Related papers  (2025-11-17T20:16:52Z)
• Speed Always Wins: A Survey on Efficient Architectures for Large Language Models [51.817121227562964]
  Large Language Models (LLMs) have delivered impressive results in language understanding, generation, reasoning, and pushes the ability boundary of multimodal models.<n> Transformer models, as the foundation of modern LLMs, offer a strong baseline with excellent scaling properties.<n>The traditional transformer architecture requires substantial computations and poses significant obstacles for large-scale training and practical deployment.
  arXiv  Detail & Related papers  (2025-08-13T14:13:46Z)
• Exploring Training and Inference Scaling Laws in Generative Retrieval [50.82554729023865]
  Generative retrieval reformulates retrieval as an autoregressive generation task, where large language models generate target documents directly from a query.<n>We systematically investigate training and inference scaling laws in generative retrieval, exploring how model size, training data scale, and inference-time compute jointly influence performance.
  arXiv  Detail & Related papers  (2025-03-24T17:59:03Z)
• ScalingNote: Scaling up Retrievers with Large Language Models for Real-World Dense Retrieval [72.2676180980573]
  Large Language Models (LLMs) have exhibited superior performance that can be leveraged for scaling up dense retrieval. We propose ScalingNote, a two-stage method to exploit the scaling potential of LLMs for retrieval while maintaining online query latency. Our two-stage scaling method outperforms end-to-end models and verifies the scaling law of dense retrieval with LLMs in industrial scenarios.
  arXiv  Detail & Related papers  (2024-11-24T09:27:43Z)
• Hierarchical Structured Neural Network: Efficient Retrieval Scaling for Large Scale Recommendation [16.21377996349377]
  We introduce the Hierarchical Structured Neural Network (HSNN), an efficient deep neural network model to learn intricate user and item interactions.<n>HSNN achieves substantial improvement in offline evaluation compared to prevailing methods.
  arXiv  Detail & Related papers  (2024-08-13T05:53:46Z)
• Diffusion-Based Neural Network Weights Generation [80.89706112736353]
  D2NWG is a diffusion-based neural network weights generation technique that efficiently produces high-performing weights for transfer learning. Our method extends generative hyper-representation learning to recast the latent diffusion paradigm for neural network weights generation. Our approach is scalable to large architectures such as large language models (LLMs), overcoming the limitations of current parameter generation techniques.
  arXiv  Detail & Related papers  (2024-02-28T08:34:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.