Beyond Single Embeddings: Capturing Diverse Targets with Multi-Query Retrieval

• URL: http://arxiv.org/abs/2511.02770v1
• Date: Tue, 04 Nov 2025 17:57:20 GMT
• Title: Beyond Single Embeddings: Capturing Diverse Targets with Multi-Query Retrieval
• Authors: Hung-Ting Chen, Xiang Liu, Shauli Ravfogel, Eunsol Choi,
• Abstract summary: We develop a new retriever architecture, emphAutoregressive emphMulti-emphEmbedding emphRetriever (AMER)<n>Our model autoregressively generates multiple query vectors, and all the predicted query vectors are used to retrieve documents from the corpus.<n>AMER presents 4 and 21% relative gains over single-embedding baselines on two datasets we evaluate on.
• Score: 45.166788898665665
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Most text retrievers generate \emph{one} query vector to retrieve relevant documents. Yet, the conditional distribution of relevant documents for the query may be multimodal, e.g., representing different interpretations of the query. We first quantify the limitations of existing retrievers. All retrievers we evaluate struggle more as the distance between target document embeddings grows. To address this limitation, we develop a new retriever architecture, \emph{A}utoregressive \emph{M}ulti-\emph{E}mbedding \emph{R}etriever (AMER). Our model autoregressively generates multiple query vectors, and all the predicted query vectors are used to retrieve documents from the corpus. We show that on the synthetic vectorized data, the proposed method could capture multiple target distributions perfectly, showing 4x better performance than single embedding model. We also fine-tune our model on real-world multi-answer retrieval datasets and evaluate in-domain. AMER presents 4 and 21\% relative gains over single-embedding baselines on two datasets we evaluate on. Furthermore, we consistently observe larger gains on the subset of dataset where the embeddings of the target documents are less similar to each other. We demonstrate the potential of using a multi-query vector retriever and open up a new direction for future work.

Related papers

• Chain of Retrieval: Multi-Aspect Iterative Search Expansion and Post-Order Search Aggregation for Full Paper Retrieval [68.71038700559195]
  Chain of Retrieval(COR) is a novel iterative framework for full-paper retrieval.<n>We present SCIBENCH, a benchmark providing both complete and segmented contexts of full papers for queries and candidates.
  arXiv  Detail & Related papers  (2025-07-14T08:41:53Z)
• Hierarchical Lexical Graph for Enhanced Multi-Hop Retrieval [22.33550491040999]
  RAG grounds large language models in external evidence, yet it still falters when answers must be pieced together across semantically distant documents.<n>We build two plug-and-play retrievers: StatementGraphRAG and TopicGraphRAG.<n>Our methods outperform naive chunk-based RAG achieving an average relative improvement of 23.1% in retrieval recall and correctness.
  arXiv  Detail & Related papers  (2025-06-09T17:58:35Z)
• Learning Refined Document Representations for Dense Retrieval via Deliberate Thinking [58.69615583599489]
  Deliberate Thinking based Retriever (Debater) is a novel approach that enhances document representations by incorporating a step-by-step thinking process.<n>Debater significantly outperforms existing methods across several retrieval benchmarks.
  arXiv  Detail & Related papers  (2025-02-18T15:56:34Z)
• Non-Parametric Memory Guidance for Multi-Document Summarization [0.0]
  We propose a retriever-guided model combined with non-parametric memory for summary generation. This model retrieves relevant candidates from a database and then generates the summary considering the candidates with a copy mechanism and the source documents. Our method is evaluated on the MultiXScience dataset which includes scientific articles.
  arXiv  Detail & Related papers  (2023-11-14T07:41:48Z)
• SPRINT: A Unified Toolkit for Evaluating and Demystifying Zero-shot Neural Sparse Retrieval [92.27387459751309]
  We provide SPRINT, a unified Python toolkit for evaluating neural sparse retrieval. We establish strong and reproducible zero-shot sparse retrieval baselines across the well-acknowledged benchmark, BEIR. We show that SPLADEv2 produces sparse representations with a majority of tokens outside of the original query and document.
  arXiv  Detail & Related papers  (2023-07-19T22:48:02Z)
• Multi-Vector Retrieval as Sparse Alignment [21.892007741798853]
  We propose a novel multi-vector retrieval model that learns sparsified pairwise alignments between query and document tokens. We learn the sparse unary saliences with entropy-regularized linear programming, which outperforms other methods to achieve sparsity. Our model often produces interpretable alignments and significantly improves its performance when from larger language models.
  arXiv  Detail & Related papers  (2022-11-02T16:49:58Z)
• UnifieR: A Unified Retriever for Large-Scale Retrieval [84.61239936314597]
  Large-scale retrieval is to recall relevant documents from a huge collection given a query. Recent retrieval methods based on pre-trained language models (PLM) can be coarsely categorized into either dense-vector or lexicon-based paradigms. We propose a new learning framework, UnifieR which unifies dense-vector and lexicon-based retrieval in one model with a dual-representing capability.
  arXiv  Detail & Related papers  (2022-05-23T11:01:59Z)
• Multi-View Document Representation Learning for Open-Domain Dense Retrieval [87.11836738011007]
  This paper proposes a multi-view document representation learning framework. It aims to produce multi-view embeddings to represent documents and enforce them to align with different queries. Experiments show our method outperforms recent works and achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2022-03-16T03:36:38Z)
• End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering [36.80395759543162]
  We present an end-to-end differentiable training method for retrieval-augmented open-domain question answering systems. We model retrieval decisions as latent variables over sets of relevant documents. Our proposed method outperforms all existing approaches of comparable size by 2-3% exact match points.
  arXiv  Detail & Related papers  (2021-06-09T19:25:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.