Benchmarking Retrieval Strategies for Biomedical Retrieval-Augmented Generation: A Controlled Empirical Study

• URL: http://arxiv.org/abs/2605.02520v1
• Date: Mon, 04 May 2026 12:21:46 GMT
• Title: Benchmarking Retrieval Strategies for Biomedical Retrieval-Augmented Generation: A Controlled Empirical Study
• Abstract summary: This paper presents a systematic empirical comparison of five retrieval strategies within a biomedical question-answering RAG pipeline.<n>All strategies share a fixed generation model (GPT-4o-mini), a common vector store (ChromaDB), and OpenAI's text-em-3-small embeddings.<n> evaluation is conducted on 250 question-answer pairs drawn from a preprocessed subset of the BioASQ benchmark.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Retrieval-Augmented Generation (RAG) offers a well-established path to grounding large language model (LLM) outputs in external knowledge, yet the question of which retrieval strategy works best in a high-stakes domain such as biomedicine has not received the controlled, multi-metric treatment it deserves. This paper presents a systematic empirical comparison of five retrieval strategies -- Dense Vector Search, Hybrid BM25 + Dense retrieval, Cross-Encoder Reranking, Multi-Query Expansion, and Maximal Marginal Relevance (MMR) -- within a biomedical question-answering RAG pipeline. All strategies share a fixed generation model (GPT-4o-mini), a common vector store (ChromaDB), and OpenAI's text-embedding-3-small embeddings, ensuring that observed differences are attributable to retrieval alone. Evaluation is conducted on 250 question-answer pairs drawn from a preprocessed subset of the BioASQ benchmark (rag-mini-bioasq) using four DeepEval metrics: contextual precision, contextual recall, faithfulness, and answer relevancy, each reported with 95% confidence intervals. A no-context ablation is included as a lower bound. Cross-Encoder Reranking achieves the best composite score (0.827) and highest contextual precision (0.852), confirming that query-document interaction yields measurable retrieval gains. Multi-Query Expansion, despite its recall-oriented design, produces the weakest contextual precision (0.671), suggesting naive query diversification introduces retrieval noise. MMR sacrifices answer relevancy for diversity, while the Dense baseline (composite 0.822) falls within 0.005 points of the top strategy. All RAG conditions dramatically outperform the no-context ablation on answer relevancy (0.658-0.701 vs. 0.287), confirming the practical value of retrieval. The full pipeline, hyperparameters, and evaluation code are publicly available.

Related papers

• VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrieval [65.89247522243959]
  VisDocAgentBench is a closed-corpus benchmark comparing static and agentic retrieval under a ranked-output contract.<n>It contains 2,375 pages from 100 documents and 120 unique-target queries balanced across direct, one-bridge, and two-bridge evidence structures.<n>A strong late-interaction visual retriever reaches 97.50% Recall@1 on direct items but 2.50% on two-bridge items, exposing the limits of query--target matching when relevance depends on corpus context.
  arXiv  Detail & Related papers  (2026-08-18T15:23:06Z)
• From BM25 to Corrective RAG: Benchmarking Retrieval Strategies for Text-and-Table Documents [0.0]
  We benchmark ten retrieval strategies spanning sparse, dense, hybrid fusion, cross-encoder reranking, query expansion, index augmentation, and adaptive retrieval.<n>We evaluate retrieval quality via Recall@k, MRR, and nDCG, and end-to-end generation quality via Number Match.
  arXiv  Detail & Related papers  (2026-04-02T07:53:40Z)
• Graph-Aware Late Chunking for Retrieval-Augmented Generation in Biomedical Literature [0.0]
  GraLC-RAG is a framework that unifies late chunking with graph-aware structural intelligence.<n>We evaluate six strategies on 2,359 IMRaD-filtered Central articles using 2,033 cross-section questions.
  arXiv  Detail & Related papers  (2026-03-23T23:22:58Z)
• Test-Time Strategies for More Efficient and Accurate Agentic RAG [58.44913384057518]
  Retrieval-Augmented Generation (RAG) systems face challenges with complex, multihop questions.<n>Such approaches can introduce inefficiencies, including repetitive retrieval of previously processed information.<n>We investigate test-time modifications to the Search-R1 pipeline to mitigate these identified shortcomings.
  arXiv  Detail & Related papers  (2026-03-12T19:18:59Z)
• Beyond Relevance: On the Relationship Between Retrieval and RAG Information Coverage [89.58253972744531]
  Retrieval-augmented generation (RAG) systems combine document retrieval with a generative model to address complex information seeking tasks like report generation.<n>We investigate whether upstream retrieval metrics can serve as reliable early indicators of the final generated response's information coverage.<n>Our findings demonstrate strong correlations between coverage-based retrieval metrics and nugget coverage in generated responses at both topic and system levels.
  arXiv  Detail & Related papers  (2026-03-09T18:20:20Z)
• BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent [74.10138164281618]
  BrowseComp-Plus is a benchmark derived from BrowseComp, employing a fixed, carefully curated corpus.<n>This benchmark allows comprehensive evaluation and disentangled analysis of deep research agents and retrieval methods.
  arXiv  Detail & Related papers  (2025-08-08T17:55:11Z)
• HySemRAG: A Hybrid Semantic Retrieval-Augmented Generation Framework for Automated Literature Synthesis and Methodological Gap Analysis [55.2480439325792]
  HySemRAG is a framework that combines Extract, Transform, Load (ETL) pipelines with Retrieval-Augmented Generation (RAG)<n>System addresses limitations in existing RAG architectures through a multi-layered approach.
  arXiv  Detail & Related papers  (2025-08-01T20:30:42Z)
• Evaluating Hybrid Retrieval Augmented Generation using Dynamic Test Sets: LiveRAG Challenge [8.680958290253914]
  We present our submission to the LiveRAG Challenge 2025, which evaluates retrieval-augmented generation (RAG) systems on dynamic test sets.<n>Our final hybrid approach combines sparse (BM25) and dense (E5) retrieval methods.<n>We demonstrate that neural re-ranking with RankLLaMA improves MAP from 0.523 to 0.797 but introduces prohibitive computational costs.
  arXiv  Detail & Related papers  (2025-06-27T21:20:43Z)
• RAGentA: Multi-Agent Retrieval-Augmented Generation for Attributed Question Answering [4.224843546370802]
  We present RAGentA, a framework for attributed question answering with large language models (LLMs)<n>With the goal of trustworthy answer generation, RAGentA focuses on optimizing answer correctness, defined by coverage and relevance to the question and faithfulness.<n>Central to the framework is a hybrid retrieval strategy that combines sparse and dense methods, improving Recall@20 by 12.5% compared to the best single retrieval model.
  arXiv  Detail & Related papers  (2025-06-20T13:37:03Z)
• Scalable Unit Harmonization in Medical Informatics via Bayesian-Optimized Retrieval and Transformer-Based Re-ranking [0.0]
  We develop a scalable methodology for harmonizing inconsistent units in large-scale clinical datasets.<n>We implement a multi-stage pipeline: filtering, identification, harmonization proposal generation, automated re-ranking, and manual validation.<n>The system achieved 83.39% precision at rank 1 and 94.66% recall at rank 5.
  arXiv  Detail & Related papers  (2025-05-01T19:09:15Z)
• ELOQ: Resources for Enhancing LLM Detection of Out-of-Scope Questions [52.33835101586687]
  We study out-of-scope questions, where the retrieved document appears semantically similar to the question but lacks the necessary information to answer it.<n>We propose a guided hallucination-based approach ELOQ to automatically generate a diverse set of out-of-scope questions from post-cutoff documents.
  arXiv  Detail & Related papers  (2024-10-18T16:11:29Z)
• Query Expansion Using Contextual Clue Sampling with Language Models [69.51976926838232]
  We propose a combination of an effective filtering strategy and fusion of the retrieved documents based on the generation probability of each context. Our lexical matching based approach achieves a similar top-5/top-20 retrieval accuracy and higher top-100 accuracy compared with the well-established dense retrieval model DPR. For end-to-end QA, the reader model also benefits from our method and achieves the highest Exact-Match score against several competitive baselines.
  arXiv  Detail & Related papers  (2022-10-13T15:18:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.