Hallucination Detection and Correction in Medical VLMs via Counter-Evidence Verification

• URL: http://arxiv.org/abs/2606.18609v1
• Date: Wed, 17 Jun 2026 02:14:44 GMT
• Title: Hallucination Detection and Correction in Medical VLMs via Counter-Evidence Verification
• Abstract summary: Vision-Language models (VLMs) reliability in medical diagnosis is challenged by trust-undermining hallucinations.<n>We propose Counter-Evidence Verification (CoEV), a training-free plug-and-play framework that detects and corrects hallucinations.<n>For hallucination detection, CoEV consistently outperforms existing methods, improving average PR-AUC and ROC-AUC.<n>For hallucination correction, it improves Micro-F1 by up to 12.5%, reduces hallucination rates by over 11.9% on medical report generation, and also boosts medical VQA accuracy.
• Score: 44.48746443825558
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language models (VLMs) reliability in medical diagnosis is challenged by trust-undermining hallucinations. Existing hallucination detection approaches mainly focus on identifying factual inconsistencies between generated text and reference data. While some studies analyze where models attend in images, they seldom verify whether such attention truly reflects the visual evidence supporting the generated text. To address this gap, we propose Co}unter-Evidence Verification (CoEV), a training-free plug-and-play framework that detects and corrects hallucinations through evidence-based factual consistency verification. CoEV performs bidirectional verification between textual assertions and visual evidence, testing whether each statement is supported by its corresponding evidence region, and assigns each statement into a four-quadrant diagnostic map capturing combinations of text factuality and visual grounding. CoEV detects hallucinated content and serves as a post hoc refinement tool, correcting hallucinations without retraining. Extensive experiments on four medical datasets show that CoEV combats hallucinations in VLMs.For hallucination detection, CoEV consistently outperforms existing methods, improving average PR-AUC and ROC-AUC by 3.0% and 3.9% absolute points respectively, with notable gains of up to 18.5% in specific VQA scenarios. For hallucination correction, it improves Micro-F1 by up to 12.5%, reduces hallucination rates by over 11.9% on medical report generation, and also boosts medical VQA accuracy. These results show that CoEV enables reliable detection and correction of hallucinations, providing clinicians with dependable, evidence-based cues for diagnosis. Code will be released upon acceptance.

Related papers

• Detecting Clinical Hallucinations in LVLMs via Counterfactual Visual Grounding Uncertainty [22.008964207160663]
  We present a vision-traceable hallucination detection framework that audits arbitrary LVLM responses via visual evidence grounding.<n>We extract visually verifiable entities and use a medical-domain-adapted Qwen-VL grounding verifier to localize each entity on the input image.<n>Experiments on multiple medical imaging modalities and LVLM backbones demonstrate that our method consistently improves hallucination detection performance.
  arXiv  Detail & Related papers  (2026-06-26T18:15:38Z)
• VIHD: Visual Intervention-based Hallucination Detection for Medical Visual Question Answering [27.827862126990553]
  Hallucinations pose risks to clinical decision-making and necessitate effective detection.<n>Existing introspective detection methods primarily perform uncertainty estimation or logical verification.<n>We propose VIHD, a Visual Intervention-based Hallucination Detection method.
  arXiv  Detail & Related papers  (2026-05-20T06:14:45Z)
• CuraView: A Multi-Agent Framework for Medical Hallucination Detection with GraphRAG-Enhanced Knowledge Verification [3.422186949568493]
  We present CuraView, a framework for sentence-level detection and evidence-grounded explanation of faithfulness hallucinations in discharge summaries.<n>CuraView constructs a GraphRAG-based knowledge graph from patient-level EHRs.<n>We evaluate CuraView on a subset of 250 patients from the Discharge-Me benchmark, with 50 patients held out for testing.
  arXiv  Detail & Related papers  (2026-05-05T08:05:31Z)
• Deterministic Hallucination Detection in Medical VQA via Confidence-Evidence Bayesian Gain [6.272224314406867]
  Multimodal large language models (MLLMs) have shown strong potential for medical Visual Question Answering (VQA)<n>They remain prone to hallucinations, defined as generating responses that contradict the input image, posing serious risks in clinical settings.<n>Current hallucination detection methods, such as Semantic Entropy (SE) and Vision-Amplified Semantic Entropy (VASE), require 10 to 20 generations per sample together with an external natural language inference model for semantic clustering, making them expensive and difficult to deploy in practice.<n>We propose a deterministic detection method that requires no sampling, no external models, and no task-
  arXiv  Detail & Related papers  (2026-03-23T08:29:53Z)
• Med-VCD: Mitigating Hallucination for Medical Large Vision Language Models through Visual Contrastive Decoding [0.8431877864777443]
  We introduce Med-VCD, a sparse visual-contrastive decoding method that mitigates hallucinations in medical LVLMs without the time overhead of secondary decoding.<n>We show that Med-VCD raises factual accuracy by an average of 13% and improves hallucination accuracy by 6% relative to baseline medical LVLMs.
  arXiv  Detail & Related papers  (2025-12-01T17:40:03Z)
• S-Chain: Structured Visual Chain-of-Thought For Medicine [81.97605645734741]
  We introduce S-Chain, the first large-scale dataset of 12,000 expert-annotated medical images with bounding boxes and structured visual CoT (SV-CoT)<n>The dataset further supports 16 languages, totaling over 700k VQA pairs for broad multilingual applicability.<n>S-Chain establishes a new benchmark for grounded medical reasoning and paves the way toward more trustworthy and explainable medical vision-language models.
  arXiv  Detail & Related papers  (2025-10-26T15:57:14Z)
• SHALE: A Scalable Benchmark for Fine-grained Hallucination Evaluation in LVLMs [52.03164192840023]
  Large Vision-Language Models (LVLMs) still suffer from hallucinations, i.e., generating content inconsistent with input or established world knowledge.<n>We propose an automated data construction pipeline that produces scalable, controllable, and diverse evaluation data.<n>We construct SHALE, a benchmark designed to assess both faithfulness and factuality hallucinations.
  arXiv  Detail & Related papers  (2025-08-13T07:58:01Z)
• Generate, but Verify: Reducing Hallucination in Vision-Language Models with Retrospective Resampling [78.78822033285938]
  Vision-Language Models (VLMs) excel at visual understanding but often suffer from visual hallucinations.<n>In this work, we introduce REVERSE, a unified framework that integrates hallucination-aware training with on-the-fly self-verification.
  arXiv  Detail & Related papers  (2025-04-17T17:59:22Z)
• MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models [82.30696225661615]
  We introduce MedHallu, the first benchmark specifically designed for medical hallucination detection.<n>We show that state-of-the-art LLMs, including GPT-4o, Llama-3.1, and medically fine-tuned UltraMedical, struggle with this binary hallucination detection task.<n>Using bidirectional entailment clustering, we show that harder-to-detect hallucinations are semantically closer to ground truth.
  arXiv  Detail & Related papers  (2025-02-20T06:33:23Z)
• Fine-grained Hallucination Detection and Editing for Language Models [109.56911670376932]
  Large language models (LMs) are prone to generate factual errors, which are often called hallucinations. We introduce a comprehensive taxonomy of hallucinations and argue that hallucinations manifest in diverse forms. We propose a novel task of automatic fine-grained hallucination detection and construct a new evaluation benchmark, FavaBench.
  arXiv  Detail & Related papers  (2024-01-12T19:02:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.