Confident but Unreliable: A Behavioral Safety Audit of Vision-Language Models on Brain MRI

• URL: http://arxiv.org/abs/2608.02790v1
• Date: Mon, 03 Aug 2026 18:42:04 GMT
• Title: Confident but Unreliable: A Behavioral Safety Audit of Vision-Language Models on Brain MRI
• Abstract summary: We use brain MRI as a controlled, high-stakes testbed for a broader failure mode in frontier multimodal systems.<n>We present an automatically graded behavioral audit and pilot study of six instruction-tuned vision-language models (VLMs) on 4,102 images.<n>Across models, answer coverage is near-complete, but verbalized-confidence calibration is poor.<n>Open-ended diagnostics show that hallucination and abstention vary separately from multiple-choice accuracy.
• Score: 1.2744523252873352
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language models (VLMs), including medical specialists, are increasingly proposed for medical imaging, yet their stated confidence is rarely evaluated separately from correctness. We use brain MRI as a controlled, high-stakes testbed for a broader failure mode in frontier multimodal systems: models can appear competent while lacking reliable self-knowledge. We present an automatically graded behavioral audit and pilot study of six instruction-tuned VLMs (five general-purpose and one medical specialist) on 4,102 images (4,032 axial/coronal/sagittal MRI slices from 250 subjects plus 70 non-brain/noise controls), with labels derived from public metadata and released expert segmentation masks rather than new human annotation. Across models, answer coverage is near-complete, but verbalized-confidence calibration is poor: ECE ranges from 0.27 to 0.40, mean confidence on incorrect answers ranges from 0.82 to 0.97, and 33-46% of answered items are high-confidence errors. The most accurate model is also the most confident on its errors, while a base/specialist family contrast suggests that medical adaptation improves tumor-presence detection without improving confidence reliability. Open-ended diagnostics further show that hallucination and abstention vary separately from multiple-choice accuracy. These findings argue that medical-image VLM evaluation should report verbalized-confidence reliability, confident error, hallucination, and abstention alongside accuracy.

Related papers

• Calibrated Triage, Not Autonomy: Confidence Estimation for Medical Vision-Language Models [4.0196226339562555]
  A vision-language model can answer a question about a medical image fluently and confidently while barely using the image.<n>We evaluate seven confidence estimators across five open-weight LVLMs and three medical visual-question-answering datasets.
  arXiv  Detail & Related papers  (2026-06-14T16:39:17Z)
• MedVIGIL: Evaluating Trustworthy Medical VLMs Under Broken Visual Evidence [24.517280048376758]
  medvigil is a 300-case evaluation suite drawn from four public medical VQA sources.<n>Every gold answer, refusal option, candidate-answer set, paraphrase, false-premise trap, ROI box, and clinical risk tier is clinician-authored.<n>We report seven correctness-conditioned audit metrics that summarise into the medvigil Composite Score (MCS)<n>The independent radiologist scores MCS 83.3 at silent-failure rate 5.8%, leaving a 14.1-point composite headroom above the strongest audited model.
  arXiv  Detail & Related papers  (2026-05-08T15:55:30Z)
• Benchmarking Multi-turn Medical Diagnosis: Hold, Lure, and Self-Correction [72.89352076103889]
  Large language models (LLMs) achieve high accuracy in medical diagnosis when all clinical information is provided in a single turn.<n>We introduce MINT, a high-fidelity, multi-turn medical diagnosis benchmark comprising 1,035 cases.<n>We uncover three persistent behavioral patterns that significantly impact diagnostic decisions.
  arXiv  Detail & Related papers  (2026-04-06T00:23:10Z)
• Multi-Agent Reasoning with Consistency Verification Improves Uncertainty Calibration in Medical MCQA [0.0]
  Multi-agent framework combines domain-specific specialist agents with Two-Phase Verification and S-Score Weighted Fusion.<n>Four specialist agents generate independent diagnoses using Qwen2.5-7B-Instruct.<n>S-scores drive a weighted fusion strategy that selects the final answer and calibrates reported confidence.
  arXiv  Detail & Related papers  (2026-03-25T16:22:53Z)
• Cerebra: A Multidisciplinary AI Board for Multimodal Dementia Characterization and Risk Assessment [56.62016795093786]
  Cerebra is an interactive multi-agent AI team that coordinates specialized agents for EHR, clinical notes, and medical imaging analysis.<n>It supports privacy-preserving deployment by operating on structured representations and remains robust when modalities are incomplete.<n>Cerebra significantly improved expert performance, increasing accuracy by 17.5 percentage points in prospective dementia risk estimation.
  arXiv  Detail & Related papers  (2026-03-23T05:46:45Z)
• Towards Reliable Medical LLMs: Benchmarking and Enhancing Confidence Estimation of Large Language Models in Medical Consultation [97.36081721024728]
  We propose the first benchmark for assessing confidence in multi-turn interaction during realistic medical consultations.<n>Our benchmark unifies three types of medical data for open-ended diagnostic generation.<n>We present MedConf, an evidence-grounded linguistic self-assessment framework.
  arXiv  Detail & Related papers  (2026-01-22T04:51:39Z)
• Evaluating Reasoning Faithfulness in Medical Vision-Language Models using Multimodal Perturbations [19.488236277427358]
  Vision-language models (VLMs) often produce chain-of-thought (CoT) explanations that sound plausible yet fail to reflect the underlying decision process.<n>We present a clinically grounded framework for chest X-ray visual question answering (VQA) that probes CoT faithfulness via controlled text and image modifications.
  arXiv  Detail & Related papers  (2025-10-13T09:28:22Z)
• EchoBench: Benchmarking Sycophancy in Medical Large Vision-Language Models [82.43729208063468]
  Recent benchmarks for medical Large Vision-Language Models (LVLMs) emphasize leaderboard accuracy, overlooking reliability and safety.<n>We study sycophancy -- models' tendency to uncritically echo user-provided information.<n>We introduce EchoBench, a benchmark to systematically evaluate sycophancy in medical LVLMs.
  arXiv  Detail & Related papers  (2025-09-24T14:09:55Z)
• Beyond Benchmarks: Dynamic, Automatic And Systematic Red-Teaming Agents For Trustworthy Medical Language Models [87.66870367661342]
  Large language models (LLMs) are used in AI applications in healthcare.<n>Red-teaming framework that continuously stress-test LLMs can reveal significant weaknesses in four safety-critical domains.<n>A suite of adversarial agents is applied to autonomously mutate test cases, identify/evolve unsafe-triggering strategies, and evaluate responses.<n>Our framework delivers an evolvable, scalable, and reliable safeguard for the next generation of medical AI.
  arXiv  Detail & Related papers  (2025-07-30T08:44:22Z)
• CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Models [92.04812189642418]
  We introduce CARES and aim to evaluate the Trustworthiness of Med-LVLMs across the medical domain. We assess the trustworthiness of Med-LVLMs across five dimensions, including trustfulness, fairness, safety, privacy, and robustness.
  arXiv  Detail & Related papers  (2024-06-10T04:07:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.