MoR: Better Handling Diverse Queries with a Mixture of Sparse, Dense, and Human Retrievers

• URL: http://arxiv.org/abs/2506.15862v1
• Date: Wed, 18 Jun 2025 20:15:23 GMT
• Title: MoR: Better Handling Diverse Queries with a Mixture of Sparse, Dense, and Human Retrievers
• Abstract summary: Retrieval-augmented Generation (RAG) is powerful, but its effectiveness hinges on which retrievers we use and how.<n>We introduce mixture of retrievers: a zero-shot, weighted combination of heterogeneous retrievers.<n>Experiments show that this mixture outperforms every individual retriever and even larger 7B models by +10.8% and +3.9% on average.
• Score: 66.30712034015009
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Retrieval-augmented Generation (RAG) is powerful, but its effectiveness hinges on which retrievers we use and how. Different retrievers offer distinct, often complementary signals: BM25 captures lexical matches; dense retrievers, semantic similarity. Yet in practice, we typically fix a single retriever based on heuristics, which fails to generalize across diverse information needs. Can we dynamically select and integrate multiple retrievers for each individual query, without the need for manual selection? In our work, we validate this intuition with quantitative analysis and introduce mixture of retrievers: a zero-shot, weighted combination of heterogeneous retrievers. Extensive experiments show that such mixtures are effective and efficient: Despite totaling just 0.8B parameters, this mixture outperforms every individual retriever and even larger 7B models by +10.8% and +3.9% on average, respectively. Further analysis also shows that this mixture framework can help incorporate specialized non-oracle human information sources as retrievers to achieve good collaboration, with a 58.9% relative performance improvement over simulated humans alone.

Related papers

• Mixture-of-Experts Language Models Can Be Strong and Efficient Retrievers [2.8881610726882827]
  Mixture-of-Experts (MoE) LLMs activate only a subset of parameters per token and are widely used to scale generative models.<n>We show that MoE retrievers outperform dense retrievers with comparable active parameter counts by up to 3.0 nDCG@10 points on BEIR.<n>One of our strongest MoE retrievers matches an 8B dense retriever with 59% fewer active parameters and 18% lower query encoding time.
  arXiv  Detail & Related papers  (2026-09-11T19:55:03Z)
• R$^3$AG: Retriever Routing for Retrieval-Augmented Generation [59.47703698994575]
  R$3$AG decomposes retriever capability into two learnable dimensions: retrieval quality and generation utility.<n>R$3$AG consistently outperforms both the best individual retrievers and state-of-the-art static routing methods.
  arXiv  Detail & Related papers  (2026-04-22T06:51:20Z)
• Revisiting RAG Retrievers: An Information Theoretic Benchmark [10.168016086591452]
  MIGRASCOPE is a Mutual Information based RAG Retriever Analysis Scope.<n>We revisit state-of-the-art retrievers and introduce principled metrics grounded in information and statistical estimation theory.<n>We show that if chosen carefully, an ensemble of retrievers outperforms any single retriever.
  arXiv  Detail & Related papers  (2026-02-25T04:19:06Z)
• SAGE: Benchmarking and Improving Retrieval for Deep Research Agents [60.53966065867568]
  We introduce SAGE, a benchmark for scientific literature retrieval comprising 1,200 queries across four scientific domains, with a 200,000 paper retrieval corpus.<n>We evaluate six deep research agents and find that all systems struggle with reasoning-intensive retrieval.<n> BM25 significantly outperforms LLM-based retrievers by approximately 30%, as existing agents generate keyword-oriented sub-queries.
  arXiv  Detail & Related papers  (2026-02-05T18:25:24Z)
• Agentic-R: Learning to Retrieve for Agentic Search [82.43426728778418]
  We propose a novel retriever training framework tailored for agentic search.<n>Unlike retrievers designed for single-turn retrieval-augmented generation (RAG), we propose to use both local query-passage relevance and global answer correctness.<n>Our retriever, termed ours, consistently outperforms strong baselines across different search agents.
  arXiv  Detail & Related papers  (2026-01-17T02:59:54Z)
• ReasonIR: Training Retrievers for Reasoning Tasks [139.54343970560103]
  ReasonIR-8B is the first retriever specifically trained for general reasoning tasks.<n>It achieves a new state-of-the-art of 29.9 nDCG@10 without reranker and 36.9 nDCG@10 with reranker on BRIGHT, a widely-used information retrieval benchmark.
  arXiv  Detail & Related papers  (2025-04-29T09:49:28Z)
• COBRA: COmBinatorial Retrieval Augmentation for Few-Shot Adaptation [5.840239260337972]
  We propose COBRA (COmBinatorial Retrieval Augmentation), which employs an alternative CMI measure that considers both diversity and similarity to a target dataset.<n> COBRA introduces negligible computational overhead to the cost of retrieval while providing significant gains in downstream model performance.
  arXiv  Detail & Related papers  (2024-12-23T16:10:07Z)
• Towards Competitive Search Relevance For Inference-Free Learned Sparse Retrievers [6.773411876899064]
  inference-free sparse models lag far behind in terms of search relevance when compared to both sparse and dense siamese models. We propose two different approaches for performance improvement. First, we introduce the IDF-aware FLOPS loss, which introduces Inverted Document Frequency (IDF) to the sparsification of representations. We find that it mitigates the negative impact of the FLOPS regularization on search relevance, allowing the model to achieve a better balance between accuracy and efficiency.
  arXiv  Detail & Related papers  (2024-11-07T03:46:43Z)
• Intrinsic Evaluation of RAG Systems for Deep-Logic Questions [2.869056892890114]
  We introduce the Overall Performance Index (OPI), an intrinsic metric to evaluate retrieval-augmented generation (RAG) mechanisms for applications involving deep-logic queries. OPI is computed as the harmonic mean of two key metrics: the Logical-Relation Correctness Ratio and the average of BERT embedding similarity scores between ground-truth and generated answers.
  arXiv  Detail & Related papers  (2024-10-03T19:25:05Z)
• Unraveling and Mitigating Retriever Inconsistencies in Retrieval-Augmented Large Language Models [25.346800371132645]
  Retrieval-augmented Large Language Models (RALMs) do not consistently outperform the original retrieval-free Language Models (LMs)<n>Our experiments reveal that this example-level performance inconsistency exists not only between retrieval-augmented and retrieval-free LM but also among different retrievers.<n>We introduce Ensemble of Retrievers (EoR), a trainable framework that can adaptively retrieve from different knowledge sources and effectively decrease unpredictable reader errors.
  arXiv  Detail & Related papers  (2024-05-31T08:22:49Z)
• SPRINT: A Unified Toolkit for Evaluating and Demystifying Zero-shot Neural Sparse Retrieval [92.27387459751309]
  We provide SPRINT, a unified Python toolkit for evaluating neural sparse retrieval. We establish strong and reproducible zero-shot sparse retrieval baselines across the well-acknowledged benchmark, BEIR. We show that SPLADEv2 produces sparse representations with a majority of tokens outside of the original query and document.
  arXiv  Detail & Related papers  (2023-07-19T22:48:02Z)
• UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation [81.80385076856304]
  We propose UPRISE, which tunes a lightweight and versatile retriever that automatically retrieves prompts for a given zero-shot task input. Specifically, we demonstrate universality in a cross-task and cross-model scenario. We show that UPRISE mitigates the hallucination problem in our experiments with ChatGPT.
  arXiv  Detail & Related papers  (2023-03-15T10:53:49Z)
• A Study on the Efficiency and Generalization of Light Hybrid Retrievers [28.08970760703045]
  We leverage an indexing-efficient dense retriever (i.e. DrBoost) and introduce a LITE retriever that further reduces the memory of DrBoost. Our Hybrid-LITE retriever saves 13X memory while maintaining 98.0% performance of the hybrid retriever of BM25 and DPR.
  arXiv  Detail & Related papers  (2022-10-04T04:22:46Z)
• Cooperative Retriever and Ranker in Deep Recommenders [75.35463122701135]
  Deep recommender systems (DRS) are intensively applied in modern web services. DRS employs a two-stage workflow: retrieval and ranking, to generate its recommendation results. It remains to explore effective collaborations between retriever and ranker.
  arXiv  Detail & Related papers  (2022-06-28T03:41:50Z)
• Towards Robust Ranker for Text Retrieval [83.15191578888188]
  A ranker plays an indispensable role in the de facto'retrieval & rerank' pipeline. A ranker plays an indispensable role in the de facto'retrieval & rerank' pipeline.
  arXiv  Detail & Related papers  (2022-06-16T10:27:46Z)
• Adversarial Retriever-Ranker for dense text retrieval [51.87158529880056]
  We present Adversarial Retriever-Ranker (AR2), which consists of a dual-encoder retriever plus a cross-encoder ranker. AR2 consistently and significantly outperforms existing dense retriever methods. This includes the improvements on Natural Questions R@5 to 77.9%(+2.1%), TriviaQA R@5 to 78.2%(+1.4), and MS-MARCO MRR@10 to 39.5%(+1.3%)
  arXiv  Detail & Related papers  (2021-10-07T16:41:15Z)
• Federated Mixture of Experts [94.25278695272874]
  FedMix is a framework that allows us to train an ensemble of specialized models. We show that users with similar data characteristics select the same members and therefore share statistical strength.
  arXiv  Detail & Related papers  (2021-07-14T14:15:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.