Confidence Calibration for Multimodal LLMs: An Empirical Study through Medical VQA

• URL: http://arxiv.org/abs/2606.19950v1
• Date: Thu, 18 Jun 2026 08:49:11 GMT
• Title: Confidence Calibration for Multimodal LLMs: An Empirical Study through Medical VQA
• Abstract summary: Multimodal Large Language Models (MLLMs) show great potential in medical tasks.<n>Their elicited confidence often misaligns with actual accuracy, potentially leading to misdiagnosis or overlooking correct advice.<n>This study presents the first comprehensive analysis of the relationship between accuracy and confidence in medical MLLMs.
• Score: 13.261644209591113
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal Large Language Models (MLLMs) show great potential in medical tasks, but their elicited confidence often misaligns with actual accuracy, potentially leading to misdiagnosis or overlooking correct advice. This study presents the first comprehensive analysis of the relationship between accuracy and confidence in medical MLLMs. It proposes a novel method that combines Multi-Strategy Fusion-Based Interrogation (MS-FBI) with auxiliary expert LLM assessment, aiming to improve confidence calibration in Medical Visual Question Answering (VQA). Experiments demonstrate that our method reduces the Expected Calibration Error (ECE) by an average of 40\% across three Medical VQA datasets, significantly enhancing MLLMs' reliability. The findings highlight the importance of domain-specific calibration for MLLMs in healthcare, offering a more trustworthy solution for AI-assisted diagnosis.

Related papers

• EHRBench: An Automated and Reliable EHR-based Benchmark for Clinical Decision Making with LLMs [51.129595320595094]
  Clinical decision-making (CDM) is central to real-world clinical, where clinicians infer diagnoses, select treatments, or anticipate future health outcomes under incomplete evidence.<n>LLM models are increasingly used to support these decisions due to strong language capabilities, broad biomedical knowledge, and efficiency.<n>The reliability of LLMs on real-world clinical decision tasks remains insufficiently understood.
  arXiv  Detail & Related papers  (2026-05-28T22:38:26Z)
• Overconfidence and Calibration in Medical VQA: Empirical Findings and Hallucination-Aware Mitigation [4.247706336083815]
  Overconfidence in vision-language models (VLMs) persists across model families and is not resolved by scaling or prompting.<n>Post-hoc calibration approaches, such as Platt scaling, reduce calibration error and consistently outperform the prompt-based strategy.<n> hallucination-aware calibration (HAC) incorporates vision-grounded hallucination detection signals as complementary inputs to refine confidence estimates.
  arXiv  Detail & Related papers  (2026-04-02T21:52:29Z)
• Towards Reliable Medical LLMs: Benchmarking and Enhancing Confidence Estimation of Large Language Models in Medical Consultation [97.36081721024728]
  We propose the first benchmark for assessing confidence in multi-turn interaction during realistic medical consultations.<n>Our benchmark unifies three types of medical data for open-ended diagnostic generation.<n>We present MedConf, an evidence-grounded linguistic self-assessment framework.
  arXiv  Detail & Related papers  (2026-01-22T04:51:39Z)
• Optimizing Medical Question-Answering Systems: A Comparative Study of Fine-Tuned and Zero-Shot Large Language Models with RAG Framework [0.0]
  We present a retrieval-augmented generation (RAG) based medical QA system that combines domain-specific knowledge retrieval with open-source LLMs to answer medical questions.<n>We fine-tune two state-of-the-art open LLMs (LLaMA2 and Falcon) using Low-Rank Adaptation (LoRA) for efficient domain specialization.<n>Our fine-tuned LLaMA2 model achieves 71.8% accuracy on PubMedQA, substantially improving over the 55.4% zero-shot baseline.
  arXiv  Detail & Related papers  (2025-12-05T16:38:47Z)
• A Multi-faceted Analysis of Cognitive Abilities: Evaluating Prompt Methods with Large Language Models on the CONSORT Checklist [1.1731001328350983]
  This study applies a behavioral and metacognitive analytic approach using an expert-validated dataset.<n>We analyze both cognitive adaptation and calibration error using metrics: Expected Error (ECE) and a baseline-normalized Relative Error (RCE)<n>Our results reveal pronounced miscalibration and overconfidence in both models, especially under clinical role-playing conditions.
  arXiv  Detail & Related papers  (2025-10-22T00:15:02Z)
• MEGAN: Mixture of Experts for Robust Uncertainty Estimation in Endoscopy Videos [2.969789372985515]
  We propose MEGAN, a Multi-Expert Gating Network that aggregates uncertainty estimates and predictions from multiple AI experts.<n>MEGAN's gating network optimally combines predictions and uncertainties from each EDL model, enhancing overall prediction confidence and calibration.<n>In large-scale prospective Ulcerative colitis (UC) clinical trial, MEGAN achieved a 3.5% improvement in F1-score and a 30.5% reduction in Expected Error (ECE) compared to existing methods.
  arXiv  Detail & Related papers  (2025-09-16T07:42:01Z)
• Prompt4Trust: A Reinforcement Learning Prompt Augmentation Framework for Clinically-Aligned Confidence Calibration in Multimodal Large Language Models [4.035767214877404]
  We introduce Prompt4Trust, the first reinforcement learning framework for prompt augmentation targeting confidence calibration in MLLMs.<n>Unlike conventional calibration techniques, Prompt4Trust specifically prioritizes aspects of calibration most critical for safe and trustworthy clinical decision-making.<n>Our framework showed promising zero-shot generalizations to larger MLLMs in our experiments.
  arXiv  Detail & Related papers  (2025-07-12T13:21:10Z)
• Calibrating Uncertainty Quantification of Multi-Modal LLMs using Grounding [48.92310906093414]
  We introduce a novel approach for calibrating uncertainty quantification (UQ) tailored for multi-modal large language models (LLMs)<n>We leverage cross-modal consistency in addition to self-consistency to improve the calibration of the multi-modal models.<n>We evaluate the proposed approach across multiple multi-modal tasks, such as medical question answering (Slake) and visual question answering (VQAv2), considering multi-modal models such as LLaVA-Med and LLaVA.
  arXiv  Detail & Related papers  (2025-04-30T19:19:21Z)
• Med-CoDE: Medical Critique based Disagreement Evaluation Framework [72.42301910238861]
  The reliability and accuracy of large language models (LLMs) in medical contexts remain critical concerns.<n>Current evaluation methods often lack robustness and fail to provide a comprehensive assessment of LLM performance.<n>We propose Med-CoDE, a specifically designed evaluation framework for medical LLMs to address these challenges.
  arXiv  Detail & Related papers  (2025-04-21T16:51:11Z)
• Structured Outputs Enable General-Purpose LLMs to be Medical Experts [50.02627258858336]
  Large language models (LLMs) often struggle with open-ended medical questions.<n>We propose a novel approach utilizing structured medical reasoning.<n>Our approach achieves the highest Factuality Score of 85.8, surpassing fine-tuned models.
  arXiv  Detail & Related papers  (2025-03-05T05:24:55Z)
• Fact or Guesswork? Evaluating Large Language Models' Medical Knowledge with Structured One-Hop Judgments [108.55277188617035]
  Large language models (LLMs) have been widely adopted in various downstream task domains, but their abilities to directly recall and apply factual medical knowledge remains under-explored.<n>We introduce the Medical Knowledge Judgment dataset (MKJ), a dataset derived from the Unified Medical Language System (UMLS), a comprehensive repository of standardized vocabularies and knowledge graphs.<n>Through a binary classification framework, MKJ evaluates LLMs' grasp of fundamental medical facts by having them assess the validity of concise, one-hop statements.
  arXiv  Detail & Related papers  (2025-02-20T05:27:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.