A Systematic Study of Biomedical Retrieval Pipeline Trade-offs in Performance and Efficiency

• URL: http://arxiv.org/abs/2604.20853v1
• Date: Mon, 23 Feb 2026 03:39:56 GMT
• Title: A Systematic Study of Biomedical Retrieval Pipeline Trade-offs in Performance and Efficiency
• Abstract summary: We study how retrieval pipeline design choices affect performance and efficiency at scale.<n>We examine retrieval over a variety of existing, public biomedical text datasets.<n>We identify several points of concrete guidance for reviewers.
• Score: 0.7734726150561086
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Retrieval systems are increasingly used in biomedical and clinical natural language processing applications, yet practical guidance for researchers building such systems is limited. In this work, we provide such guidance through an empirical study of how retrieval pipeline design choices affect performance and efficiency at scale. In particular, we examine retrieval over a variety of existing, public biomedical text datasets, leveraging a variety of disparate types of queries, including exam-style questions, conversational medical queries, community-asked questions, and non-question formulations across various retrieval pipeline settings spanning corpus selection, chunk granularity, and vector index configuration. Retrieval results are judged using a robust, win-rate comparison assessment via an LLM-as-a-judge setting with human validation. Across these experiments, we identify several points of concrete guidance for reviewers, including the superiority of corpus aggregation for absolute retrieval quality, and the emergence of MedRAG/pubmed as the Pareto-optimal singleton corpus under graph-based (HNSW) indexing, appropriate chunking strategies, and FAISS indexing choices that offer the best trade-offs in speed and efficiency.

Related papers

• Health System Scale Semantic Search Across Unstructured Clinical Notes [1.599023522858371]
  We deploy a semantic search system at a large children's hospital indexing 166 million clinical notes from 1.68 million patients.<n>The system delivers sub-latency query latency (median 237 ms single-user, 451 ms 20-usertoken) with monthly costs of approximately USD 4,000.
  arXiv  Detail & Related papers  (2026-04-28T13:09:48Z)
• A Systematic Study of Retrieval Pipeline Design for Retrieval-Augmented Medical Question Answering [0.0]
  Large language models (LLMs) have demonstrated strong capabilities in medical question answering.<n>Retrieval-augmented generation (RAG) addresses this limitation by integrating external knowledge retrieval into the reasoning process.<n>This study presents a systematic evaluation of RAG-based medical question answering using the MedQA USMLE benchmark and a structured textbook-based knowledge corpus.
  arXiv  Detail & Related papers  (2026-04-08T16:37:22Z)
• Retrieval-Augmented Generation Assistant for Anatomical Pathology Laboratories [2.125786478579275]
  This study proposes and evaluates a Retrieval-Augmented Generation (RAG) assistant tailored to Anatomical Pathology (AP) laboratories.<n>We curated a novel corpus of 99 AP protocols from a Portuguese healthcare institution and constructed 323 question-answer pairs.<n>Retrieval-Augmented Generation assistant provides technicians with context-grounded answers to protocol-related queries.
  arXiv  Detail & Related papers  (2025-12-08T22:44:57Z)
• Demo: Guide-RAG: Evidence-Driven Corpus Curation for Retrieval-Augmented Generation in Long COVID [3.716027375029187]
  We developed and evaluated six Retrieval-Augmented Generation (RAG) corpus configurations for Long COVID (LC) clinical question answering.<n>Our RAG corpus configuration combining clinical guidelines with high-quality systematic reviews consistently outperformed both narrow single-guideline approaches and large-scale literature databases.
  arXiv  Detail & Related papers  (2025-10-17T16:05:52Z)
• Distilling a Small Utility-Based Passage Selector to Enhance Retrieval-Augmented Generation [110.610512800947]
  Retrieval-augmented generation (RAG) enhances large language models (LLMs) by incorporating retrieved information.<n>In RAG, the emphasis has shifted to utility, which considers the usefulness of passages for generating accurate answers.<n>Our approach focuses on utility-based selection rather than ranking, enabling dynamic passage selection tailored to specific queries without the need for fixed thresholds.<n>Our experiments demonstrate that utility-based selection provides a flexible and cost-effective solution for RAG, significantly reducing computational costs while improving answer quality.
  arXiv  Detail & Related papers  (2025-07-25T09:32:29Z)
• CliniQ: A Multi-faceted Benchmark for Electronic Health Record Retrieval with Semantic Match Assessment [11.815222175336695]
  We introduce a novel public EHR retrieval benchmark, CliniQ, to address this gap.<n>We build our benchmark upon 1,000 discharge summary notes along with the ICD codes and prescription labels from MIMIC-III.<n>We conduct a comprehensive evaluation of various retrieval methods, ranging from conventional exact match to popular dense retrievers.
  arXiv  Detail & Related papers  (2025-02-10T08:33:47Z)
• Comprehensive and Practical Evaluation of Retrieval-Augmented Generation Systems for Medical Question Answering [70.44269982045415]
  Retrieval-augmented generation (RAG) has emerged as a promising approach to enhance the performance of large language models (LLMs) We introduce Medical Retrieval-Augmented Generation Benchmark (MedRGB) that provides various supplementary elements to four medical QA datasets. Our experimental results reveals current models' limited ability to handle noise and misinformation in the retrieved documents.
  arXiv  Detail & Related papers  (2024-11-14T06:19:18Z)
• SeRTS: Self-Rewarding Tree Search for Biomedical Retrieval-Augmented Generation [50.26966969163348]
  Large Language Models (LLMs) have shown great potential in the biomedical domain with the advancement of retrieval-augmented generation (RAG) Existing retrieval-augmented approaches face challenges in addressing diverse queries and documents, particularly for medical knowledge queries. We propose Self-Rewarding Tree Search (SeRTS) based on Monte Carlo Tree Search (MCTS) and a self-rewarding paradigm.
  arXiv  Detail & Related papers  (2024-06-17T06:48:31Z)
• Development and validation of a natural language processing algorithm to pseudonymize documents in the context of a clinical data warehouse [53.797797404164946]
  The study highlights the difficulties faced in sharing tools and resources in this domain. We annotated a corpus of clinical documents according to 12 types of identifying entities. We build a hybrid system, merging the results of a deep learning model as well as manual rules.
  arXiv  Detail & Related papers  (2023-03-23T17:17:46Z)
• SPeC: A Soft Prompt-Based Calibration on Performance Variability of Large Language Model in Clinical Notes Summarization [50.01382938451978]
  We introduce a model-agnostic pipeline that employs soft prompts to diminish variance while preserving the advantages of prompt-based summarization. Experimental findings indicate that our method not only bolsters performance but also effectively curbs variance for various language models.
  arXiv  Detail & Related papers  (2023-03-23T04:47:46Z)
• A method for comparing multiple imputation techniques: a case study on the U.S. National COVID Cohort Collaborative [1.259457977936316]
  We numerically evaluate strategies for handling missing data in the context of statistical analysis. Our approach could effectively highlight the most valid and performant missing-data handling strategy.
  arXiv  Detail & Related papers  (2022-06-13T19:49:54Z)
• Self-supervised Answer Retrieval on Clinical Notes [68.87777592015402]
  We introduce CAPR, a rule-based self-supervision objective for training Transformer language models for domain-specific passage matching. We apply our objective in four Transformer-based architectures: Contextual Document Vectors, Bi-, Poly- and Cross-encoders. We report that CAPR outperforms strong baselines in the retrieval of domain-specific passages and effectively generalizes across rule-based and human-labeled passages.
  arXiv  Detail & Related papers  (2021-08-02T10:42:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.