Can Large Language Models Self-Correct in Medical Question Answering? An Exploratory Study

• URL: http://arxiv.org/abs/2604.00261v2
• Date: Thu, 02 Apr 2026 14:18:08 GMT
• Title: Can Large Language Models Self-Correct in Medical Question Answering? An Exploratory Study
• Abstract summary: Large language models (LLMs) have achieved strong performance on medical question answering (medical QA)<n>Chain-of-thought (CoT) prompting has further improved results by eliciting explicit intermediate reasoning.<n>Self-reflective prompting has been widely claimed to enhance model reliability by prompting LLMs to critique and revise their own reasoning.
• Score: 10.405961618570847
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) have achieved strong performance on medical question answering (medical QA), and chain-of-thought (CoT) prompting has further improved results by eliciting explicit intermediate reasoning; meanwhile, self-reflective (self-corrective) prompting has been widely claimed to enhance model reliability by prompting LLMs to critique and revise their own reasoning, yet its effectiveness in safety-critical medical settings remains unclear. In this work, we conduct an exploratory analysis of self-reflective reasoning for medical multiple-choice question answering: using GPT-4o and GPT-4o-mini, we compare standard CoT prompting with an iterative self-reflection loop and track how predictions evolve across reflection steps on three widely used medical QA benchmarks (MedQA, HeadQA, and PubMedQA). We analyze whether self-reflection leads to error correction, error persistence, or the introduction of new errors. Our results show that self-reflective prompting does not consistently improve accuracy and its impact is highly dataset- and model-dependent: it yields modest gains on MedQA but provides limited or negative benefits on HeadQA and PubMedQA, and increasing the number of reflection steps does not guarantee better performance. These findings highlight a gap between reasoning transparency and reasoning correctness, suggesting that self-reflective reasoning is better viewed as an analytical tool for understanding model behavior rather than a standalone solution for improving medical QA reliability.

Related papers

• Beyond the Leaderboard: Design Lessons for Trustworthy Multimodal VQA [4.080704298118247]
  multimodal AI must combine visual and textual evidence while remaining reliable and interpretable.<n>We analyze design choices across nine documented systems for question answering and explanation quality.<n>The findings support trustworthy multimodal healthcare AI based on data fusion, explainability, and resilient evaluation.
  arXiv  Detail & Related papers  (2026-07-16T17:38:19Z)
• MedCausalX: Adaptive Causal Reasoning with Self-Reflection for Trustworthy Medical Vision-Language Models [10.466505116993451]
  Existing medical chain-of-thought models lack explicit mechanisms to represent and enforce causal reasoning.<n>We propose MedCausalX, an end-to-end framework explicitly models causal reasoning chains in medical VLMs.<n>We show that MedCausalX consistently outperforms state-of-the-art methods, improving diagnostic consistency by +5.4 points, reducing hallucination by over 10 points, and attaining top spatial grounding IoU.
  arXiv  Detail & Related papers  (2026-03-24T11:28:15Z)
• Recursive Think-Answer Process for LLMs and VLMs [54.52289112197118]
  We propose an efficient Recursive Think-Answer Process (R-TAP)<n>R-TAP enables models to engage in iterative reasoning cycles and generate more accurate answers.<n>We show that R-TAP-enhanced models consistently outperform conventional single-pass methods.
  arXiv  Detail & Related papers  (2026-03-02T17:20:10Z)
• On Calibration of Large Language Models: From Response To Capability [66.59139960234326]
  Large language models (LLMs) are widely deployed as general-purpose problem solvers.<n>We introduce capability calibration, which targets the model's expected accuracy on a query.<n>Our results demonstrate that capability-calibrated confidence improves pass@$k$ prediction and inference budget allocation.
  arXiv  Detail & Related papers  (2026-02-14T01:07:45Z)
• Self-MedRAG: a Self-Reflective Hybrid Retrieval-Augmented Generation Framework for Reliable Medical Question Answering [39.146761527401424]
  Self-MedRAG is a self-reflective hybrid framework designed to mimic the iterative hypothesis-verification process of clinical reasoning.<n>It integrates a hybrid retrieval strategy, combining sparse (BM25) and dense (Contriever) retrievers via Reciprocal Rank Fusion.<n>It employs a generator to produce answers with supporting rationales, which are then assessed by a lightweight self-reflection module.
  arXiv  Detail & Related papers  (2026-01-08T02:56:04Z)
• MedAlign: A Synergistic Framework of Multimodal Preference Optimization and Federated Meta-Cognitive Reasoning [52.064286116035134]
  We develop MedAlign, a framework to ensure visually accurate LVLM responses for Medical Visual Question Answering (Med-VQA)<n>We first propose a multimodal Direct Preference Optimization (mDPO) objective to align preference learning with visual context.<n>We then design a Retrieval-Aware Mixture-of-Experts (RA-MoE) architecture that utilizes image and text similarity to route queries to a specialized and context-augmented LVLM.
  arXiv  Detail & Related papers  (2025-10-24T02:11:05Z)
• MedReflect: Teaching Medical LLMs to Self-Improve via Reflective Correction [23.71420855072473]
  We introduce MedReflect, a framework designed to inspire large language models with a physician-like reflective thinking mode.<n>We demonstrate that MedReflect enables cost-efficient medical dataset construction.<n>Our results provide evidence that LLMs can learn to solve specialized medical problems via self-reflection and self-improve.
  arXiv  Detail & Related papers  (2025-10-04T06:00:48Z)
• AdaThink-Med: Medical Adaptive Thinking with Uncertainty-Guided Length Calibration [4.33177021777927]
  We propose AdaThink-Med, an end-to-end framework designed to enhance adaptive thinking ability in medical reasoning models.<n>AdaThink-Med first generates multiple candidate outputs for each question, evaluates the correctness and uncertainty of each candidate, and then estimates problem difficulty via an uncertainty-guided length calibration module.<n>On six public medical QA benchmarks, AdaThink-Med achieves up to 6.4x length reduction on average while retaining performance with only minimal degradation.
  arXiv  Detail & Related papers  (2025-09-29T10:13:55Z)
• GEMeX-RMCoT: An Enhanced Med-VQA Dataset for Region-Aware Multimodal Chain-of-Thought Reasoning [60.03671205298294]
  Medical visual question answering aims to support clinical decision-making by enabling models to answer natural language questions based on medical images.<n>Current methods still suffer from limited answer reliability and poor interpretability.<n>This work first proposes a Region-Aware Multimodal Chain-of-Thought dataset, in which the process of producing an answer is preceded by a sequence of intermediate reasoning steps.
  arXiv  Detail & Related papers  (2025-06-22T08:09:58Z)
• Structured Outputs Enable General-Purpose LLMs to be Medical Experts [50.02627258858336]
  Large language models (LLMs) often struggle with open-ended medical questions.<n>We propose a novel approach utilizing structured medical reasoning.<n>Our approach achieves the highest Factuality Score of 85.8, surpassing fine-tuned models.
  arXiv  Detail & Related papers  (2025-03-05T05:24:55Z)
• Small Language Models Need Strong Verifiers to Self-Correct Reasoning [69.94251699982388]
  Self-correction has emerged as a promising solution to boost the reasoning performance of large language models (LLMs) This work explores whether small (= 13B) language models (LMs) have the ability of self-correction on reasoning tasks with minimal inputs from stronger LMs.
  arXiv  Detail & Related papers  (2024-04-26T03:41:28Z)
• When Hindsight is Not 20/20: Testing Limits on Reflective Thinking in Large Language Models [15.781930031346105]
  Self-reflection enhances performance in TruthfulQA, but adversely affects results in HotpotQA. We find that self-reflection shows the most benefit when models are less likely to be correct initially, and when overall question difficulty is higher. Based on our findings, we propose guidelines for decisions on when to implement self-reflection.
  arXiv  Detail & Related papers  (2024-04-14T02:47:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.