Align then Train: Efficient Retrieval Adapter Learning

• URL: http://arxiv.org/abs/2604.03403v1
• Date: Fri, 03 Apr 2026 19:04:42 GMT
• Title: Align then Train: Efficient Retrieval Adapter Learning
• Abstract summary: We propose a label-efficient framework that trains retrieval adapters in two stages: self-supervised alignment and supervised adaptation.<n>Experiments on the MAIR benchmark, spanning 126 retrieval tasks across 6 domains, show that ERA improves retrieval in low-label settings.
• Score: 16.855103318831073
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dense retrieval systems increasingly need to handle complex queries. In many realistic settings, users express intent through long instructions or task-specific descriptions, while target documents remain relatively simple and static. This asymmetry creates a retrieval mismatch: understanding queries may require strong reasoning and instruction-following, whereas efficient document indexing favors lightweight encoders. Existing retrieval systems often address this mismatch by directly improving the embedding model, but fine-tuning large embedding models to better follow such instructions is computationally expensive, memory-intensive, and operationally burdensome. To address this challenge, we propose Efficient Retrieval Adapter (ERA), a label-efficient framework that trains retrieval adapters in two stages: self-supervised alignment and supervised adaptation. Inspired by the pre-training and supervised fine-tuning stages of LLMs, ERA first aligns the embedding spaces of a large query embedder and a lightweight document embedder, and then uses limited labeled data to adapt the query-side representation, bridging both the representation gap between embedding models and the semantic gap between complex queries and simple documents without re-indexing the corpus. Experiments on the MAIR benchmark, spanning 126 retrieval tasks across 6 domains, show that ERA improves retrieval in low-label settings, outperforms methods that rely on larger amounts of labeled data, and effectively combines stronger query embedders with weaker document embedders across domains.

Related papers

• AnnoRetrieve: Efficient Structured Retrieval for Unstructured Document Analysis [11.689256498133446]
  We propose AnnoRetrieve, a novel retrieval paradigm that shifts from embeddings to structured annotations.<n>Our system replaces expensive vector comparisons with lightweight structured queries over automatically induced schemas.<n>AnoRetrieve establishes a new paradigm for cost-effective, precise, and scalable document analysis through intelligent structuring.
  arXiv  Detail & Related papers  (2026-04-03T03:34:19Z)
• LITTA: Late-Interaction and Test-Time Alignment for Visually-Grounded Multimodal Retrieval [0.0]
  LITTA is a query-expansion-centric retrieval framework for evidence page retrieval.<n>Given a user query, LITTA generates complementary query variants using a large language model.<n>Candidates from expanded queries are aggregated through reciprocal rank fusion to improve evidence coverage.
  arXiv  Detail & Related papers  (2026-03-10T13:25:39Z)
• Succeeding at Scale: Automated Multi-Retriever Fusion and Query-Side Adaptation for Multi-Tenant Search [22.080200394842123]
  DevRev Search is a passage retrieval benchmark for technical customer support constructed through a fully automatic pipeline.<n>We propose a practical textbfIndex-Preserving Adaptation strategy, by fine-tuning only the query encoder via Low-Rank Adaptation.<n>Our experiments on DevRev Search and SciFact demonstrate that targeting specific transformer layers in the query encoder yields optimal quality-efficiency trade-offs.
  arXiv  Detail & Related papers  (2026-01-08T06:44:40Z)
• BookRAG: A Hierarchical Structure-aware Index-based Approach for Retrieval-Augmented Generation on Complex Documents [11.158307125677375]
  Retrieval-Augmented Generation (RAG) queries highly relevant information from external complex documents.<n>We introduce BookRAG, a novel RAG approach targeted for documents with a hierarchical structure.<n>BookRAG achieves state-of-the-art performance, significantly outperforming baselines in both retrieval recall and QA accuracy.
  arXiv  Detail & Related papers  (2025-12-03T03:40:49Z)
• URaG: Unified Retrieval and Generation in Multimodal LLMs for Efficient Long Document Understanding [55.45331924836242]
  We present URaG, a framework that Unifies Retrieval and Generation within a single MLLM.<n>We show that URaG achieves state-of-the-art performance while reducing computational overhead by 44-56%.
  arXiv  Detail & Related papers  (2025-11-13T17:54:09Z)
• Cluster-based Adaptive Retrieval: Dynamic Context Selection for RAG Applications [8.946586077722822]
  Cluster-based Adaptive Retrieval (CAR) is an algorithm that determines the optimal number of documents by analyzing the clustering patterns of ordered query-document similarity distances.<n>CAR consistently picks the optimal retrieval depth and achieves the highest TES score, outperforming every fixed top-k baseline.
  arXiv  Detail & Related papers  (2025-10-02T05:11:12Z)
• Reasoning-enhanced Query Understanding through Decomposition and Interpretation [87.56450566014625]
  ReDI is a Reasoning-enhanced approach for query understanding through Decomposition and Interpretation.<n>We compiled a large-scale dataset of real-world complex queries from a major search engine.<n> Experiments on BRIGHT and BEIR demonstrate that ReDI consistently surpasses strong baselines in both sparse and dense retrieval paradigms.
  arXiv  Detail & Related papers  (2025-09-08T10:58:42Z)
• Improving Document Retrieval Coherence for Semantically Equivalent Queries [63.97649988164166]
  We propose a variation of the Multi-Negative Ranking loss for training DR that improves the coherence of models in retrieving the same documents.<n>The loss penalizes discrepancies between the top-k ranked documents retrieved for diverse but semantic equivalent queries.
  arXiv  Detail & Related papers  (2025-08-11T13:34:59Z)
• QID: Efficient Query-Informed ViTs in Data-Scarce Regimes for OCR-free Visual Document Understanding [53.69841526266547]
  Fine-tuning a pre-trained Vision-Language Model with new datasets often falls short in optimizing the vision encoder.<n>We introduce QID, a novel, streamlined, architecture-preserving approach that integrates query embeddings into the vision encoder.
  arXiv  Detail & Related papers  (2025-04-03T18:47:16Z)
• Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity [59.57065228857247]
  Retrieval-augmented Large Language Models (LLMs) have emerged as a promising approach to enhancing response accuracy in several tasks, such as Question-Answering (QA) We propose a novel adaptive QA framework, that can dynamically select the most suitable strategy for (retrieval-augmented) LLMs based on the query complexity. We validate our model on a set of open-domain QA datasets, covering multiple query complexities, and show that ours enhances the overall efficiency and accuracy of QA systems.
  arXiv  Detail & Related papers  (2024-03-21T13:52:30Z)
• Augmenting Document Representations for Dense Retrieval with Interpolation and Perturbation [49.940525611640346]
  Document Augmentation for dense Retrieval (DAR) framework augments the representations of documents with their Dense Augmentation and perturbations. We validate the performance of DAR on retrieval tasks with two benchmark datasets, showing that the proposed DAR significantly outperforms relevant baselines on the dense retrieval of both the labeled and unlabeled documents.
  arXiv  Detail & Related papers  (2022-03-15T09:07:38Z)
• Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback [29.719150565643965]
  This paper proposes ANCE-PRF, a new query encoder that uses pseudo relevance feedback (PRF) to improve query representations for dense retrieval. ANCE-PRF uses a BERT encoder that consumes the query and the top retrieved documents from a dense retrieval model, ANCE, and it learns to produce better query embeddings directly from relevance labels. Analysis shows that the PRF encoder effectively captures the relevant and complementary information from PRF documents, while ignoring the noise with its learned attention mechanism.
  arXiv  Detail & Related papers  (2021-08-30T18:10:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.