Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?

• URL: http://arxiv.org/abs/2110.06918v1
• Date: Wed, 13 Oct 2021 17:56:19 GMT
• Title: Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?
• Abstract summary: We introduce the Salient Phrase Aware Retriever (SPAR), a dense retriever with the lexical matching capacity of a sparse model. In particular, we show that a dense retriever Lambda can be trained to imitate a sparse one, and SPAR is built by augmenting a standard dense retriever with Lambda.
• Score: 15.618287626892174
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite their recent popularity and well known advantages, dense retrievers still lag behind sparse methods such as BM25 in their ability to reliably match salient phrases and rare entities in the query. It has been argued that this is an inherent limitation of dense models. We disprove this claim by introducing the Salient Phrase Aware Retriever (SPAR), a dense retriever with the lexical matching capacity of a sparse model. In particular, we show that a dense retriever {\Lambda} can be trained to imitate a sparse one, and SPAR is built by augmenting a standard dense retriever with {\Lambda}. When evaluated on five open-domain question answering datasets and the MS MARCO passage retrieval task, SPAR sets a new state of the art for dense and sparse retrievers and can match or exceed the performance of more complicated dense-sparse hybrid systems.

Related papers

• Latent Terms: Dense Retrievers Contain Trivially Extractable BM25-ready Zipfian Vocabularies [1.938658977168075]
  We propose Latent Terms, a method revealing that models trained for dense retrieval learn representations that can trivially be decomposed into sparse features.<n>When trained on frozen retrievers, Sparse Autoencoders without any retrieval-specific adjustments extract a latent vocabulary with approximately Zipfian collection statistics.<n>Latent Terms is able to match or outperform single-vector scoring methods from its own base model as well as comparable SPLADE variants.
  arXiv  Detail & Related papers  (2026-05-28T05:36:37Z)
• SAGE: Benchmarking and Improving Retrieval for Deep Research Agents [60.53966065867568]
  We introduce SAGE, a benchmark for scientific literature retrieval comprising 1,200 queries across four scientific domains, with a 200,000 paper retrieval corpus.<n>We evaluate six deep research agents and find that all systems struggle with reasoning-intensive retrieval.<n> BM25 significantly outperforms LLM-based retrievers by approximately 30%, as existing agents generate keyword-oriented sub-queries.
  arXiv  Detail & Related papers  (2026-02-05T18:25:24Z)
• Exp4Fuse: A Rank Fusion Framework for Enhanced Sparse Retrieval using Large Language Model-based Query Expansion [0.0]
  Large Language Models (LLMs) have shown potential in generating hypothetical documents for query expansion.<n>We introduce a novel fusion ranking framework, Exp4Fuse, which enhances the performance of sparse retrievers.
  arXiv  Detail & Related papers  (2025-06-05T08:44:34Z)
• Hierarchical Re-ranker Retriever (HRR) [2.6527830618420967]
  This paper introduces the Hierarchical Re-ranker Retriever (HRR), a framework designed to achieve both fine-grained and high-level context retrieval. In HRR, documents are split into sentence-level and intermediate-level (512 tokens) chunks to maximize vector-search quality for both short and broad queries.
  arXiv  Detail & Related papers  (2025-03-04T08:43:19Z)
• Mistral-SPLADE: LLMs for better Learned Sparse Retrieval [7.652738829153342]
  We propose to use decoder-only model for learning semantic keyword expansion. We use Mistral as the backbone to develop our Learned Sparse Retriever similar to SPLADE. Our experiments support the hypothesis that a sparse retrieval model based on decoder only large language model (LLM) surpasses the performance of existing LSR systems.
  arXiv  Detail & Related papers  (2024-08-20T18:21:54Z)
• Unsupervised Multilingual Dense Retrieval via Generative Pseudo Labeling [32.10366004426449]
  This paper introduces UMR, an Unsupervised dense Multilingual Retriever trained without any paired data. We propose a two-stage framework which iteratively improves the performance of multilingual dense retrievers.
  arXiv  Detail & Related papers  (2024-03-06T07:49:06Z)
• Dense X Retrieval: What Retrieval Granularity Should We Use? [56.90827473115201]
  Often-overlooked design choice is the retrieval unit in which the corpus is indexed, e.g. document, passage, or sentence. We introduce a novel retrieval unit, proposition, for dense retrieval. Experiments reveal that indexing a corpus by fine-grained units such as propositions significantly outperforms passage-level units in retrieval tasks.
  arXiv  Detail & Related papers  (2023-12-11T18:57:35Z)
• CONVERSER: Few-Shot Conversational Dense Retrieval with Synthetic Data Generation [32.10366004426449]
  We propose CONVERSER, a framework for training conversational dense retrievers with at most 6 examples of in-domain dialogues. We utilize the in-context learning capability of large language models to generate conversational queries given a passage in the retrieval corpus. Experimental results on conversational retrieval benchmarks OR-QuAC and TREC CAsT 19 show that the proposed CONVERSER achieves comparable performance to fully-supervised models.
  arXiv  Detail & Related papers  (2023-09-13T06:40:24Z)
• SPRINT: A Unified Toolkit for Evaluating and Demystifying Zero-shot Neural Sparse Retrieval [92.27387459751309]
  We provide SPRINT, a unified Python toolkit for evaluating neural sparse retrieval. We establish strong and reproducible zero-shot sparse retrieval baselines across the well-acknowledged benchmark, BEIR. We show that SPLADEv2 produces sparse representations with a majority of tokens outside of the original query and document.
  arXiv  Detail & Related papers  (2023-07-19T22:48:02Z)
• Large Language Models are Strong Zero-Shot Retriever [89.16756291653371]
  We propose a simple method that applies a large language model (LLM) to large-scale retrieval in zero-shot scenarios. Our method, the Language language model as Retriever (LameR), is built upon no other neural models but an LLM.
  arXiv  Detail & Related papers  (2023-04-27T14:45:55Z)
• Bridging the Training-Inference Gap for Dense Phrase Retrieval [104.4836127502683]
  Building dense retrievers requires a series of standard procedures, including training and validating neural models. In this paper, we explore how the gap between training and inference in dense retrieval can be reduced. We propose an efficient way of validating dense retrievers using a small subset of the entire corpus.
  arXiv  Detail & Related papers  (2022-10-25T00:53:06Z)
• A Study on the Efficiency and Generalization of Light Hybrid Retrievers [28.08970760703045]
  We leverage an indexing-efficient dense retriever (i.e. DrBoost) and introduce a LITE retriever that further reduces the memory of DrBoost. Our Hybrid-LITE retriever saves 13X memory while maintaining 98.0% performance of the hybrid retriever of BM25 and DPR.
  arXiv  Detail & Related papers  (2022-10-04T04:22:46Z)
• LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrieval [68.85686621130111]
  We propose to make a dense retriever align a well-performing lexicon-aware representation model. We evaluate our model on three public benchmarks, which shows that with a comparable lexicon-aware retriever as the teacher, our proposed dense model can bring consistent and significant improvements.
  arXiv  Detail & Related papers  (2022-08-29T15:09:28Z)
• UnifieR: A Unified Retriever for Large-Scale Retrieval [84.61239936314597]
  Large-scale retrieval is to recall relevant documents from a huge collection given a query. Recent retrieval methods based on pre-trained language models (PLM) can be coarsely categorized into either dense-vector or lexicon-based paradigms. We propose a new learning framework, UnifieR which unifies dense-vector and lexicon-based retrieval in one model with a dual-representing capability.
  arXiv  Detail & Related papers  (2022-05-23T11:01:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.