PubMed Reasoner: Dynamic Reasoning-based Retrieval for Evidence-Grounded Biomedical Question Answering

• URL: http://arxiv.org/abs/2603.27335v1
• Date: Sat, 28 Mar 2026 16:41:57 GMT
• Title: PubMed Reasoner: Dynamic Reasoning-based Retrieval for Evidence-Grounded Biomedical Question Answering
• Abstract summary: We introduce PubMed Reasoner, a biomedical QA agent composed of three stages.<n>Self-critic query refinement evaluates MeSH terms for coverage, alignment, and redundancy to enhance PubMed queries.<n> reflective retrieval processes articles in batches until sufficient evidence is gathered.<n>Evidence-grounded response generation produces answers with explicit citations.
• Score: 12.717987334814692
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Trustworthy biomedical question answering (QA) systems must not only provide accurate answers but also justify them with current, verifiable evidence. Retrieval-augmented approaches partially address this gap but lack mechanisms to iteratively refine poor queries, whereas self-reflection methods kick in only after full retrieval is completed. In this context, we introduce PubMed Reasoner, a biomedical QA agent composed of three stages: self-critic query refinement evaluates MeSH terms for coverage, alignment, and redundancy to enhance PubMed queries based on partial (metadata) retrieval; reflective retrieval processes articles in batches until sufficient evidence is gathered; and evidence-grounded response generation produces answers with explicit citations. PubMed Reasoner with a GPT-4o backbone achieves 78.32% accuracy on PubMedQA, slightly surpassing human experts, and showing consistent gains on MMLU Clinical Knowledge. Moreover, LLM-as-judge evaluations prefer our responses across: reasoning soundness, evidence grounding, clinical relevance, and trustworthiness. By orchestrating retrieval-first reasoning over authoritative sources, our approach provides practical assistance to clinicians and biomedical researchers while controlling compute and token costs.

Related papers

• When Retrieval Helps and Distracts: Evaluating Evidence-Generating LLMs for Biomedical Claim Verification [0.17188280334580197]
  We study this evidence-generation setting on CARE-XAI, a unified benchmark spanning five biomedical and health fact-checking sources.<n>Bio-GRACE shows that retrieval utility is source-dependent, motivates selective retrieval, and exposes why retrieval recall and lexical evidence overlap are insufficient for biomedical fact-checking.
  arXiv  Detail & Related papers  (2026-08-02T17:43:28Z)
• ReMedi: Reasoner for Medical Clinical Prediction [70.84466325266068]
  We present ReMedi, a framework for improving clinical outcome prediction from EHR.<n>ReMedi generates rationale-answer pairs using a challenging sample regeneration mechanism for complex clinical questions.<n> Experiments on multiple EHR prediction tasks demonstrate substantial gains of up to 19.9 percent over state-of-the-art baselines in terms of F1 score.
  arXiv  Detail & Related papers  (2026-05-02T14:44:49Z)
• Same Verdict, Different Reasons: LLM-as-a-Judge and Clinician Disagreement on Medical Chatbot Completeness [49.2667937337333]
  We stress-test this assumption for detecting incomplete patient-facing medical responses.<n>We evaluate three granularities (General-Likert, Analytical-Rubric, Dynamic-Checklist) and three backbone models across two clinician-annotated datasets.
  arXiv  Detail & Related papers  (2026-03-26T19:01:55Z)
• Towards Reliable Medical LLMs: Benchmarking and Enhancing Confidence Estimation of Large Language Models in Medical Consultation [97.36081721024728]
  We propose the first benchmark for assessing confidence in multi-turn interaction during realistic medical consultations.<n>Our benchmark unifies three types of medical data for open-ended diagnostic generation.<n>We present MedConf, an evidence-grounded linguistic self-assessment framework.
  arXiv  Detail & Related papers  (2026-01-22T04:51:39Z)
• Self-MedRAG: a Self-Reflective Hybrid Retrieval-Augmented Generation Framework for Reliable Medical Question Answering [39.146761527401424]
  Self-MedRAG is a self-reflective hybrid framework designed to mimic the iterative hypothesis-verification process of clinical reasoning.<n>It integrates a hybrid retrieval strategy, combining sparse (BM25) and dense (Contriever) retrievers via Reciprocal Rank Fusion.<n>It employs a generator to produce answers with supporting rationales, which are then assessed by a lightweight self-reflection module.
  arXiv  Detail & Related papers  (2026-01-08T02:56:04Z)
• MedTrust-RAG: Evidence Verification and Trust Alignment for Biomedical Question Answering [21.855579328680246]
  We propose MedTrust-Guided Iterative RAG, a framework designed to enhance factual consistency and hallucinations in medical QA.<n>First, it enforces citation-aware reasoning by requiring all generated content to be explicitly grounded in retrieved medical documents.<n>Second, it employs an iterative retrieval-verification process, where a verification agent assesses evidence adequacy.
  arXiv  Detail & Related papers  (2025-10-16T07:59:11Z)
• Simulating Viva Voce Examinations to Evaluate Clinical Reasoning in Large Language Models [51.91760712805404]
  We introduce VivaBench, a benchmark for evaluating sequential clinical reasoning in large language models (LLMs)<n>Our dataset consists of 1762 physician-curated clinical vignettes structured as interactive scenarios that simulate a (oral) examination in medical training.<n>Our analysis identified several failure modes that mirror common cognitive errors in clinical practice.
  arXiv  Detail & Related papers  (2025-10-11T16:24:35Z)
• Introducing Answered with Evidence -- a framework for evaluating whether LLM responses to biomedical questions are founded in evidence [1.3250161978024673]
  Large language models (LLMs) for biomedical question answering raise concerns about the accuracy and evidentiary support of their responses.<n>We analyzed thousands of physician-submitted questions using a comparative pipeline that included: (1) Alexandria, fka the Atropos Evidence Library, a retrieval-augmented generation (RAG) system based on novel observational studies, and (2) two PubMed-based retrieval-augmented systems (System and Perplexity)<n>We found that PubMed-based systems provided evidence-supported answers for approximately 44% of questions, while the novel evidence source did so for about 50%.
  arXiv  Detail & Related papers  (2025-06-30T18:00:52Z)
• Decide less, communicate more: On the construct validity of end-to-end fact-checking in medicine [59.604255567812714]
  We show how experts verify real claims from social media by synthesizing medical evidence.<n>Difficulties connecting claims in the wild to scientific evidence in the form of clinical trials.<n>We argue that fact-checking should be approached and evaluated as an interactive communication problem.
  arXiv  Detail & Related papers  (2025-06-25T22:58:08Z)
• Improving Reliability and Explainability of Medical Question Answering through Atomic Fact Checking in Retrieval-Augmented LLMs [15.61511109105186]
  Large language models (LLMs) exhibit extensive medical knowledge but are prone to hallucinations and inaccurate citations.<n>Current methods, such as Retrieval Augmented Generation, partially address these issues by grounding answers in source documents.<n>We introduce a novel atomic fact-checking framework designed to enhance the reliability and explainability of LLMs.
  arXiv  Detail & Related papers  (2025-05-30T17:33:07Z)
• Structured Outputs Enable General-Purpose LLMs to be Medical Experts [50.02627258858336]
  Large language models (LLMs) often struggle with open-ended medical questions.<n>We propose a novel approach utilizing structured medical reasoning.<n>Our approach achieves the highest Factuality Score of 85.8, surpassing fine-tuned models.
  arXiv  Detail & Related papers  (2025-03-05T05:24:55Z)
• MedCoT: Medical Chain of Thought via Hierarchical Expert [48.91966620985221]
  This paper presents MedCoT, a novel hierarchical expert verification reasoning chain method.<n>It is designed to enhance interpretability and accuracy in biomedical imaging inquiries.<n> Experimental evaluations on four standard Med-VQA datasets demonstrate that MedCoT surpasses existing state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-12-18T11:14:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.