A Systematic Study of Retrieval Pipeline Design for Retrieval-Augmented Medical Question Answering

• URL: http://arxiv.org/abs/2604.07274v1
• Date: Wed, 08 Apr 2026 16:37:22 GMT
• Title: A Systematic Study of Retrieval Pipeline Design for Retrieval-Augmented Medical Question Answering
• Abstract summary: Large language models (LLMs) have demonstrated strong capabilities in medical question answering.<n>Retrieval-augmented generation (RAG) addresses this limitation by integrating external knowledge retrieval into the reasoning process.<n>This study presents a systematic evaluation of RAG-based medical question answering using the MedQA USMLE benchmark and a structured textbook-based knowledge corpus.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) have demonstrated strong capabilities in medical question answering; however, purely parametric models often suffer from knowledge gaps and limited factual grounding. Retrieval-augmented generation (RAG) addresses this limitation by integrating external knowledge retrieval into the reasoning process. Despite increasing interest in RAG-based medical systems, the impact of individual retrieval components on performance remains insufficiently understood. This study presents a systematic evaluation of retrieval-augmented medical question answering using the MedQA USMLE benchmark and a structured textbook-based knowledge corpus. We analyze the interaction between language models, embedding models, retrieval strategies, query reformulation, and cross-encoder reranking within a unified experimental framework comprising forty configurations. Results show that retrieval augmentation significantly improves zero-shot medical question answering performance. The best-performing configuration was dense retrieval with query reformulation and reranking achieved 60.49% accuracy. Domain-specialized language models were also found to better utilize retrieved medical evidence than general-purpose models. The analysis further reveals a clear tradeoff between retrieval effectiveness and computational cost, with simpler dense retrieval configurations providing strong performance while maintaining higher throughput. All experiments were conducted on a single consumer-grade GPU, demonstrating that systematic evaluation of retrieval-augmented medical QA systems can be performed under modest computational resources.

Related papers

• Proactive Reasoning-with-Retrieval Framework for Medical Multimodal Large Language Models [15.530083855947987]
  We propose the first Multimodal Medical Reasoning-with-Retrieval framework, Med-RwR.<n>Med-RwR actively retrieves external knowledge by querying observed symptoms or domain-specific medical concepts during reasoning.<n> Evaluation on various public medical benchmarks demonstrates Med-RwR's significant improvements over baseline models.
  arXiv  Detail & Related papers  (2025-10-21T05:18:18Z)
• RAR$^2$: Retrieval-Augmented Medical Reasoning via Thought-Driven Retrieval [25.425621641226815]
  Large Language Models (LLMs) have shown promising performance on diverse medical benchmarks.<n>RAG has emerged as a key approach for mitigating knowledge gaps and hallucinations by incorporating external medical information.<n>We propose RAR$2$, a joint learning framework that improves Reasoning-Augmented Retrieval and Retrieval-Augmented Reasoning.
  arXiv  Detail & Related papers  (2025-09-24T05:35:57Z)
• Biomedical Literature Q&A System Using Retrieval-Augmented Generation (RAG) [0.0]
  This work presents a Biomedical Literature Question Answering (Q&A) system based on a Retrieval-Augmented Generation architecture.<n>The system integrates diverse sources, including PubMed articles, curated Q&A datasets, and medical encyclopedias.<n>The system supports both general medical queries and domain-specific tasks, with a focused evaluation on breast cancer literature.
  arXiv  Detail & Related papers  (2025-09-05T21:29:52Z)
• End-to-End Agentic RAG System Training for Traceable Diagnostic Reasoning [52.12425911708585]
  Deep-DxSearch is an agentic RAG system trained end-to-end with reinforcement learning (RL)<n>In Deep-DxSearch, we first construct a large-scale medical retrieval corpus comprising patient records and reliable medical knowledge sources.<n> Experiments demonstrate that our end-to-end RL training framework consistently outperforms prompt-engineering and training-free RAG approaches.
  arXiv  Detail & Related papers  (2025-08-21T17:42:47Z)
• Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications [59.721265428780946]
  Large Language Models (LLMs) in medicine have enabled impressive capabilities, yet a critical gap remains in their ability to perform systematic, transparent, and verifiable reasoning.<n>This paper provides the first systematic review of this emerging field.<n>We propose a taxonomy of reasoning enhancement techniques, categorized into training-time strategies and test-time mechanisms.
  arXiv  Detail & Related papers  (2025-08-01T14:41:31Z)
• Optimizing Retrieval-Augmented Generation of Medical Content for Spaced Repetition Learning [0.46603287532620746]
  This paper presents a pipeline employing comments generation for Poland's State Retrieval Examination (PES) based on verified resources.<n>The system integrates these generated comments and source documents with a spaced repetition learning algorithm to enhance knowledge retention.<n> Rigorous evaluation by medical annotators demonstrates improvements in key metrics such as document relevance, credibility, and logical coherence of generated content.
  arXiv  Detail & Related papers  (2025-02-23T20:56:31Z)
• Knowledge Graph-Driven Retrieval-Augmented Generation: Integrating Deepseek-R1 with Weaviate for Advanced Chatbot Applications [45.935798913942904]
  We propose an innovative framework that combines structured biomedical knowledge with large language models (LLMs)<n>Our system develops a thorough knowledge graph by identifying and refining causal relationships and named entities from medical abstracts related to age-related macular degeneration (AMD)<n>Using a vector-based retrieval process and a locally deployed language model, our framework produces responses that are both contextually relevant and verifiable, with direct references to clinical evidence.
  arXiv  Detail & Related papers  (2025-02-16T12:52:28Z)
• Comprehensive and Practical Evaluation of Retrieval-Augmented Generation Systems for Medical Question Answering [70.44269982045415]
  Retrieval-augmented generation (RAG) has emerged as a promising approach to enhance the performance of large language models (LLMs) We introduce Medical Retrieval-Augmented Generation Benchmark (MedRGB) that provides various supplementary elements to four medical QA datasets. Our experimental results reveals current models' limited ability to handle noise and misinformation in the retrieved documents.
  arXiv  Detail & Related papers  (2024-11-14T06:19:18Z)
• SeRTS: Self-Rewarding Tree Search for Biomedical Retrieval-Augmented Generation [50.26966969163348]
  Large Language Models (LLMs) have shown great potential in the biomedical domain with the advancement of retrieval-augmented generation (RAG) Existing retrieval-augmented approaches face challenges in addressing diverse queries and documents, particularly for medical knowledge queries. We propose Self-Rewarding Tree Search (SeRTS) based on Monte Carlo Tree Search (MCTS) and a self-rewarding paradigm.
  arXiv  Detail & Related papers  (2024-06-17T06:48:31Z)
• CorpusBrain: Pre-train a Generative Retrieval Model for Knowledge-Intensive Language Tasks [62.22920673080208]
  Single-step generative model can dramatically simplify the search process and be optimized in end-to-end manner. We name the pre-trained generative retrieval model as CorpusBrain as all information about the corpus is encoded in its parameters without the need of constructing additional index.
  arXiv  Detail & Related papers  (2022-08-16T10:22:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.