A Study on the Efficiency and Generalization of Light Hybrid Retrievers

• URL: http://arxiv.org/abs/2210.01371v2
• Date: Tue, 23 May 2023 09:45:30 GMT
• Title: A Study on the Efficiency and Generalization of Light Hybrid Retrievers
• Abstract summary: We leverage an indexing-efficient dense retriever (i.e. DrBoost) and introduce a LITE retriever that further reduces the memory of DrBoost. Our Hybrid-LITE retriever saves 13X memory while maintaining 98.0% performance of the hybrid retriever of BM25 and DPR.
• Score: 28.08970760703045
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hybrid retrievers can take advantage of both sparse and dense retrievers. Previous hybrid retrievers leverage indexing-heavy dense retrievers. In this work, we study "Is it possible to reduce the indexing memory of hybrid retrievers without sacrificing performance"? Driven by this question, we leverage an indexing-efficient dense retriever (i.e. DrBoost) and introduce a LITE retriever that further reduces the memory of DrBoost. LITE is jointly trained on contrastive learning and knowledge distillation from DrBoost. Then, we integrate BM25, a sparse retriever, with either LITE or DrBoost to form light hybrid retrievers. Our Hybrid-LITE retriever saves 13X memory while maintaining 98.0% performance of the hybrid retriever of BM25 and DPR. In addition, we study the generalization capacity of our light hybrid retrievers on out-of-domain dataset and a set of adversarial attacks datasets. Experiments showcase that light hybrid retrievers achieve better generalization performance than individual sparse and dense retrievers. Nevertheless, our analysis shows that there is a large room to improve the robustness of retrievers, suggesting a new research direction.

Related papers

• Mixture-of-Experts Language Models Can Be Strong and Efficient Retrievers [2.8881610726882827]
  Mixture-of-Experts (MoE) LLMs activate only a subset of parameters per token and are widely used to scale generative models.<n>We show that MoE retrievers outperform dense retrievers with comparable active parameter counts by up to 3.0 nDCG@10 points on BEIR.<n>One of our strongest MoE retrievers matches an 8B dense retriever with 59% fewer active parameters and 18% lower query encoding time.
  arXiv  Detail & Related papers  (2026-09-11T19:55:03Z)
• SAGE: Benchmarking and Improving Retrieval for Deep Research Agents [60.53966065867568]
  We introduce SAGE, a benchmark for scientific literature retrieval comprising 1,200 queries across four scientific domains, with a 200,000 paper retrieval corpus.<n>We evaluate six deep research agents and find that all systems struggle with reasoning-intensive retrieval.<n> BM25 significantly outperforms LLM-based retrievers by approximately 30%, as existing agents generate keyword-oriented sub-queries.
  arXiv  Detail & Related papers  (2026-02-05T18:25:24Z)
• Agentic-R: Learning to Retrieve for Agentic Search [82.43426728778418]
  We propose a novel retriever training framework tailored for agentic search.<n>Unlike retrievers designed for single-turn retrieval-augmented generation (RAG), we propose to use both local query-passage relevance and global answer correctness.<n>Our retriever, termed ours, consistently outperforms strong baselines across different search agents.
  arXiv  Detail & Related papers  (2026-01-17T02:59:54Z)
• MoR: Better Handling Diverse Queries with a Mixture of Sparse, Dense, and Human Retrievers [66.30712034015009]
  Retrieval-augmented Generation (RAG) is powerful, but its effectiveness hinges on which retrievers we use and how.<n>We introduce mixture of retrievers: a zero-shot, weighted combination of heterogeneous retrievers.<n>Experiments show that this mixture outperforms every individual retriever and even larger 7B models by +10.8% and +3.9% on average.
  arXiv  Detail & Related papers  (2025-06-18T20:15:23Z)
• ReasonIR: Training Retrievers for Reasoning Tasks [139.54343970560103]
  ReasonIR-8B is the first retriever specifically trained for general reasoning tasks. It achieves a new state-of-the-art of 29.9 nDCG@10 without reranker and 36.9 nDCG@10 with reranker on BRIGHT, a widely-used information retrieval benchmark.
  arXiv  Detail & Related papers  (2025-04-29T09:49:28Z)
• DRAMA: Diverse Augmentation from Large Language Models to Smaller Dense Retrievers [86.54316283425001]
  Large language models (LLMs) have demonstrated strong effectiveness and robustness while fine-tuned as dense retrievers. LLMs offer better efficiency, but they often fail to generalize effectively with limited supervised fine-tuning data. We introduce DRAMA, a training framework that leverages LLMs to train smaller generalizable dense retrievers.
  arXiv  Detail & Related papers  (2025-02-25T18:59:07Z)
• Towards Competitive Search Relevance For Inference-Free Learned Sparse Retrievers [6.773411876899064]
  inference-free sparse models lag far behind in terms of search relevance when compared to both sparse and dense siamese models. We propose two different approaches for performance improvement. First, we introduce the IDF-aware FLOPS loss, which introduces Inverted Document Frequency (IDF) to the sparsification of representations. We find that it mitigates the negative impact of the FLOPS regularization on search relevance, allowing the model to achieve a better balance between accuracy and efficiency.
  arXiv  Detail & Related papers  (2024-11-07T03:46:43Z)
• Exploring Demonstration Retrievers in RAG for Coding Tasks: Yeas and Nays! [6.34946724864899]
  This paper systematically evaluates the efficiency-effectiveness trade-off of retrievers across three coding tasks. We show that while BM25 excels in effectiveness, it suffers in efficiency as the knowledge base grows beyond 1000 entries. In large-scale retrieval, efficiency differences become more pronounced, with approximate dense retrievers offering the greatest gains.
  arXiv  Detail & Related papers  (2024-10-12T22:31:01Z)
• Boot and Switch: Alternating Distillation for Zero-Shot Dense Retrieval [50.47192086219752]
  $texttABEL$ is a simple but effective unsupervised method to enhance passage retrieval in zero-shot settings. By either fine-tuning $texttABEL$ on labelled data or integrating it with existing supervised dense retrievers, we achieve state-of-the-art results.
  arXiv  Detail & Related papers  (2023-11-27T06:22:57Z)
• UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation [81.80385076856304]
  We propose UPRISE, which tunes a lightweight and versatile retriever that automatically retrieves prompts for a given zero-shot task input. Specifically, we demonstrate universality in a cross-task and cross-model scenario. We show that UPRISE mitigates the hallucination problem in our experiments with ChatGPT.
  arXiv  Detail & Related papers  (2023-03-15T10:53:49Z)
• Bridging the Training-Inference Gap for Dense Phrase Retrieval [104.4836127502683]
  Building dense retrievers requires a series of standard procedures, including training and validating neural models. In this paper, we explore how the gap between training and inference in dense retrieval can be reduced. We propose an efficient way of validating dense retrievers using a small subset of the entire corpus.
  arXiv  Detail & Related papers  (2022-10-25T00:53:06Z)
• LaPraDoR: Unsupervised Pretrained Dense Retriever for Zero-Shot Text Retrieval [55.097573036580066]
  Experimental results show that LaPraDoR achieves state-of-the-art performance compared with supervised dense retrieval models. Compared to re-ranking, our lexicon-enhanced approach can be run in milliseconds (22.5x faster) while achieving superior performance.
  arXiv  Detail & Related papers  (2022-03-11T18:53:12Z)
• Learning to Retrieve Passages without Supervision [58.31911597824848]
  Dense retrievers for open-domain question answering (ODQA) have been shown to achieve impressive performance by training on large datasets of question-passage pairs. We investigate whether dense retrievers can be learned in a self-supervised fashion, and applied effectively without any annotations.
  arXiv  Detail & Related papers  (2021-12-14T19:18:08Z)
• Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One? [15.618287626892174]
  We introduce the Salient Phrase Aware Retriever (SPAR), a dense retriever with the lexical matching capacity of a sparse model. In particular, we show that a dense retriever Lambda can be trained to imitate a sparse one, and SPAR is built by augmenting a standard dense retriever with Lambda.
  arXiv  Detail & Related papers  (2021-10-13T17:56:19Z)
• Adversarial Retriever-Ranker for dense text retrieval [51.87158529880056]
  We present Adversarial Retriever-Ranker (AR2), which consists of a dual-encoder retriever plus a cross-encoder ranker. AR2 consistently and significantly outperforms existing dense retriever methods. This includes the improvements on Natural Questions R@5 to 77.9%(+2.1%), TriviaQA R@5 to 78.2%(+1.4), and MS-MARCO MRR@10 to 39.5%(+1.3%)
  arXiv  Detail & Related papers  (2021-10-07T16:41:15Z)
• Is Retriever Merely an Approximator of Reader? [27.306407064073177]
  We show that the reader and the retriever are complementary to each other even in terms of accuracy only. We propose to distill the reader into the retriever so that the retriever absorbs the strength of the reader while keeping its own benefit.
  arXiv  Detail & Related papers  (2020-10-21T13:40:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.