Efficient Retrieval Scaling with Hierarchical Indexing for Large Scale Recommendation

• URL: http://arxiv.org/abs/2604.12965v1
• Date: Tue, 14 Apr 2026 16:59:03 GMT
• Title: Efficient Retrieval Scaling with Hierarchical Indexing for Large Scale Recommendation
• Abstract summary: This paper explores whether it is possible to learn a hierarchical organization over the memory of foundational retrieval models.<n>We propose jointly learning a hierarchical index using cross-attention and residual quantization for large-scale retrieval models.<n>We present our real-world deployment at Meta, supporting daily advertisement recommendations for billions of Facebook and Instagram users.
• Score: 18.266884490896228
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The increase in data volume, computational resources, and model parameters during training has led to the development of numerous large-scale industrial retrieval models for recommendation tasks. However, effectively and efficiently deploying these large-scale foundational retrieval models remains a critical challenge that has not been fully addressed. Common quick-win solutions for deploying these massive models include relying on offline computations (such as cached user dictionaries) or distilling large models into smaller ones. Yet, both approaches fall short of fully leveraging the representational and inference capabilities of foundational models. In this paper, we explore whether it is possible to learn a hierarchical organization over the memory of foundational retrieval models. Such a hierarchical structure would enable more efficient search by reducing retrieval costs while preserving exactness. To achieve this, we propose jointly learning a hierarchical index using cross-attention and residual quantization for large-scale retrieval models. We also present its real-world deployment at Meta, supporting daily advertisement recommendations for billions of Facebook and Instagram users. Interestingly, we discovered that the intermediate nodes in the learned index correspond to a small set of high-quality data. Fine-tuning the model on this set further improves inference performance, and concretize the concept of "test-time training" within the recommendation system domain. We demonstrate these findings using both internal and public datasets with strong baseline comparisons and hope they contribute to the community's efforts in developing the next generation of foundational retrieval models.

Related papers

• The Case Against Generation for Retrieval: Discriminative Language Models as Effective Retrievers [49.488975839992044]
  Large Language Models (LLMs) have emerged as powerful assets for recommender systems.<n> deploying them as generative recommenders or zero-shot rankers at web-scale remains bottlenecked by prohibitive computational overhead and grounding challenges.<n>In this paper, we revitalize the classic, highly efficient two-tower retrieval architecture by adapting LLMs as semantic representation backbones rather than generative engines.
  arXiv  Detail & Related papers  (2026-07-28T06:47:53Z)
• A Survey on Efficient Vision-Language-Action Models [153.11669266922993]
  Vision-Language-Action models (VLAs) represent a significant frontier in embodied intelligence, aiming to bridge digital knowledge with physical-world interaction.<n>Motivated by the urgent need to address these challenges, this survey presents the first comprehensive review of Efficient Vision-Language-Action models.
  arXiv  Detail & Related papers  (2025-10-27T17:57:33Z)
• Exploring Training and Inference Scaling Laws in Generative Retrieval [50.82554729023865]
  Generative retrieval reformulates retrieval as an autoregressive generation task, where large language models generate target documents directly from a query.<n>We systematically investigate training and inference scaling laws in generative retrieval, exploring how model size, training data scale, and inference-time compute jointly influence performance.
  arXiv  Detail & Related papers  (2025-03-24T17:59:03Z)
• A Collaborative Ensemble Framework for CTR Prediction [73.59868761656317]
  We propose a novel framework, Collaborative Ensemble Training Network (CETNet), to leverage multiple distinct models. Unlike naive model scaling, our approach emphasizes diversity and collaboration through collaborative learning. We validate our framework on three public datasets and a large-scale industrial dataset from Meta.
  arXiv  Detail & Related papers  (2024-11-20T20:38:56Z)
• UpLIF: An Updatable Self-Tuning Learned Index Framework [4.077820670802213]
  UpLIF is an adaptive self-tuning learned index that adjusts the model to accommodate incoming updates. We also introduce the concept of balanced model adjustment, which determines the model's inherent properties.
  arXiv  Detail & Related papers  (2024-08-07T22:30:43Z)
• Reinforcement Learning with Generative Models for Compact Support Sets [10.041289551532804]
  We propose a framework utilizing reinforcement learning as a control for foundation models. Our framework produced excellent results, increasing classification accuracy by significant margins for no additional labelling or data cost.
  arXiv  Detail & Related papers  (2024-04-25T02:48:16Z)
• Back to Basics: A Simple Recipe for Improving Out-of-Domain Retrieval in Dense Encoders [63.28408887247742]
  We study whether training procedures can be improved to yield better generalization capabilities in the resulting models. We recommend a simple recipe for training dense encoders: Train on MSMARCO with parameter-efficient methods, such as LoRA, and opt for using in-batch negatives unless given well-constructed hard negatives.
  arXiv  Detail & Related papers  (2023-11-16T10:42:58Z)
• CorpusBrain: Pre-train a Generative Retrieval Model for Knowledge-Intensive Language Tasks [62.22920673080208]
  Single-step generative model can dramatically simplify the search process and be optimized in end-to-end manner. We name the pre-trained generative retrieval model as CorpusBrain as all information about the corpus is encoded in its parameters without the need of constructing additional index.
  arXiv  Detail & Related papers  (2022-08-16T10:22:49Z)
• Which Model to Transfer? Finding the Needle in the Growing Haystack [27.660318887140203]
  We provide a formalization of this problem through a familiar notion of regret. We show that both task-agnostic and task-aware methods can yield high regret. We then propose a simple and efficient hybrid search strategy which outperforms the existing approaches.
  arXiv  Detail & Related papers  (2020-10-13T14:00:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.