Vision-Amplified Semantic Entropy for Hallucination Detection in Medical Visual Question Answering

• URL: http://arxiv.org/abs/2503.20504v1
• Date: Wed, 26 Mar 2025 12:45:34 GMT
• Title: Vision-Amplified Semantic Entropy for Hallucination Detection in Medical Visual Question Answering
• Abstract summary: Multimodal large language models (MLLMs) have demonstrated significant potential in medical Visual Question Answering (VQA)<n>They remain prone to hallucinations-incorrect responses that contradict input images, posing substantial risks in clinical decision-making.<n>Current hallucination detection methods, especially semantic entropy (SE), have demonstrated promising hallucination detection capacity for LLMs.<n>We propose Vision Amplified Semantic Entropy (VASE), which incorporates weak image transformations and amplifies the impact of visual input.
• Score: 42.116697933322875
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal large language models (MLLMs) have demonstrated significant potential in medical Visual Question Answering (VQA). Yet, they remain prone to hallucinations-incorrect responses that contradict input images, posing substantial risks in clinical decision-making. Detecting these hallucinations is essential for establishing trust in MLLMs among clinicians and patients, thereby enabling their real-world adoption. Current hallucination detection methods, especially semantic entropy (SE), have demonstrated promising hallucination detection capacity for LLMs. However, adapting SE to medical MLLMs by incorporating visual perturbations presents a dilemma. Weak perturbations preserve image content and ensure clinical validity, but may be overlooked by medical MLLMs, which tend to over rely on language priors. In contrast, strong perturbations can distort essential diagnostic features, compromising clinical interpretation. To address this issue, we propose Vision Amplified Semantic Entropy (VASE), which incorporates weak image transformations and amplifies the impact of visual input, to improve hallucination detection in medical VQA. We first estimate the semantic predictive distribution under weak visual transformations to preserve clinical validity, and then amplify visual influence by contrasting this distribution with that derived from a distorted image. The entropy of the resulting distribution is estimated as VASE. Experiments on two medical open-ended VQA datasets demonstrate that VASE consistently outperforms existing hallucination detection methods.

Related papers

• HIVE: Understanding Post-Hallucination Reasoning in Vision Language Models [70.67112531913669]
  Hallucinations in vision language models are commonly treated as semantic errors.<n>Prior work mainly focuses on detecting or suppressing hallucinations at generation time.<n>We study Post Hallucination Reasoning (PHR), the stage in which hallucinated semantics enter the model's inference context.
  arXiv  Detail & Related papers  (2026-07-08T15:02:11Z)
• Detecting Clinical Hallucinations in LVLMs via Counterfactual Visual Grounding Uncertainty [22.008964207160663]
  We present a vision-traceable hallucination detection framework that audits arbitrary LVLM responses via visual evidence grounding.<n>We extract visually verifiable entities and use a medical-domain-adapted Qwen-VL grounding verifier to localize each entity on the input image.<n>Experiments on multiple medical imaging modalities and LVLM backbones demonstrate that our method consistently improves hallucination detection performance.
  arXiv  Detail & Related papers  (2026-06-26T18:15:38Z)
• VIHD: Visual Intervention-based Hallucination Detection for Medical Visual Question Answering [27.827862126990553]
  Hallucinations pose risks to clinical decision-making and necessitate effective detection.<n>Existing introspective detection methods primarily perform uncertainty estimation or logical verification.<n>We propose VIHD, a Visual Intervention-based Hallucination Detection method.
  arXiv  Detail & Related papers  (2026-05-20T06:14:45Z)
• VCE: A zero-cost hallucination mitigation method of LVLMs via visual contrastive editing [70.82867621856968]
  Large vision-language models (LVLMs) frequently suffer from Object Hallucination (OH)<n>Recent studies suggest that the hallucination problem may stem from language priors.<n>We propose Visual Contrastive Editing (VCE), a novel post-hoc method that identifies and suppresses hallucinatory tendencies.
  arXiv  Detail & Related papers  (2026-04-21T12:40:07Z)
• V-Loop: Visual Logical Loop Verification for Hallucination Detection in Medical Visual Question Answering [8.994256499686863]
  We propose a training-free and plug-and-play framework for hallucination detection in medical visual question answering.<n>V-Loop forms a visually grounded logical loop to verify factual correctness.<n>It consistently outperforms existing introspective methods, remains highly efficient, and further boosts uncertainty-based approaches when used in combination.
  arXiv  Detail & Related papers  (2026-01-26T07:46:41Z)
• On Epistemic Uncertainty of Visual Tokens for Object Hallucinations in Large Vision-Language Models [27.228426342808486]
  We argue that uncertain visual tokens within the vision encoder (VE) is a key factor that contributes to object hallucination.<n>We propose a simple yet effective strategy to mitigate object hallucination by modifying the VE only.
  arXiv  Detail & Related papers  (2025-10-10T05:12:52Z)
• Exploring and Mitigating Fawning Hallucinations in Large Language Models [52.444712272909435]
  We analyze fawning hallucinations in various natural language processing tasks.<n>We tailor the so-termed contrastive decoding method for fawning-hallucination mitigation.
  arXiv  Detail & Related papers  (2025-08-31T14:29:54Z)
• Two Causes, Not One: Rethinking Omission and Fabrication Hallucinations in MLLMs [31.601057368065877]
  Existing methods, based on the flawed assumption that omission and fabrication hallucinations share a common cause, often reduce omissions only to trigger more fabrications.<n>In this work, we overturn this view by demonstrating that omission hallucinations arise from insufficient confidence when mapping perceived visual features to linguistic expressions.<n>We propose the Visual-Semantic Attention Potential Field, a conceptual framework that reveals how visual evidence to infer the presence or absence of objects.
  arXiv  Detail & Related papers  (2025-08-30T05:47:41Z)
• MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM [58.2298313720146]
  Multimodal hallucinations are multi-sourced and arise from diverse causes.<n>Existing benchmarks fail to adequately distinguish between perception-induced hallucinations and reasoning-induced hallucinations.
  arXiv  Detail & Related papers  (2025-05-30T05:54:36Z)
• A Comprehensive Analysis for Visual Object Hallucination in Large Vision-Language Models [30.037505914306504]
  Vision-Language Models (LVLMs) demonstrate remarkable capabilities in multimodal tasks.<n>LVLMs generate inaccurate visual object-related information based on the query input, potentially leading to misinformation and concerns about safety and reliability.<n>In this paper, we analyze each component of LLaVA-like LVLMs to identify potential sources of error and their impact.
  arXiv  Detail & Related papers  (2025-05-04T01:47:58Z)
• Poison as Cure: Visual Noise for Mitigating Object Hallucinations in LVMs [7.920981206857122]
  Large vision-language models (LVMs) extend large language models (LLMs) with visual perception capabilities.<n>A major challenge compromising their reliability is object hallucination that LVMs may generate plausible but factually inaccurate information.<n>We propose a novel visual adversarial perturbation (VAP) method to mitigate this hallucination issue.
  arXiv  Detail & Related papers  (2025-01-31T14:31:00Z)
• Towards a Systematic Evaluation of Hallucinations in Large-Vision Language Models [57.58426038241812]
  Large Vision-Language Models (LVLMs) have demonstrated remarkable performance in complex multimodal tasks.<n>These models still suffer from hallucinations when required to implicitly recognize or infer diverse visual entities from images.<n>We propose a novel visual question answering (VQA) benchmark that employs contextual reasoning prompts as hallucination attacks.
  arXiv  Detail & Related papers  (2024-12-29T23:56:01Z)
• Combating Multimodal LLM Hallucination via Bottom-Up Holistic Reasoning [151.4060202671114]
  multimodal large language models (MLLMs) have shown unprecedented capabilities in advancing vision-language tasks.<n>This paper introduces a novel bottom-up reasoning framework to address hallucinations in MLLMs.<n>Our framework systematically addresses potential issues in both visual and textual inputs by verifying and integrating perception-level information with cognition-level commonsense knowledge.
  arXiv  Detail & Related papers  (2024-12-15T09:10:46Z)
• CATCH: Complementary Adaptive Token-level Contrastive Decoding to Mitigate Hallucinations in LVLMs [74.36850397755572]
  CATCH addresses issues related to visual defects that cause diminished fine-grained feature perception and cumulative hallucinations in open-ended scenarios. It is applicable to various visual question-answering tasks without requiring any specific data or prior knowledge, and generalizes robustly to new tasks without additional training.
  arXiv  Detail & Related papers  (2024-11-19T18:27:31Z)
• Reducing Hallucinations in Vision-Language Models via Latent Space Steering [34.1755878632361]
  Hallucination poses a challenge to the deployment of large vision-language models (LVLMs) in applications. We introduce Visual and Textual Intervention (VTI), a novel technique designed to reduce hallucinations by steering latent space representations during inference to enhance the stability of vision features.
  arXiv  Detail & Related papers  (2024-10-21T08:42:30Z)
• Mitigating Hallucinations of Large Language Models in Medical Information Extraction via Contrastive Decoding [92.32881381717594]
  We introduce ALternate Contrastive Decoding (ALCD) to solve hallucination issues in medical information extraction tasks. ALCD demonstrates significant improvements in resolving hallucination issues compared to conventional decoding methods.
  arXiv  Detail & Related papers  (2024-10-21T07:19:19Z)
• From Pixels to Tokens: Revisiting Object Hallucinations in Large Vision-Language Models [15.401221354325672]
  Hallucinations in large vision models (LVLMs) are a significant challenge, i.e., generating objects that are not presented in the visual input. Recent studies often attribute hallucinations to a lack of understanding of visual input, yet ignore a more fundamental issue: the model's inability to extract or decouple visual features. In this paper, we revisit the hallucinations in LVLMs from an architectural perspective, investigating whether the primary cause lies in the visual encoder (feature extraction) or the modal alignment module (feature decoupling)
  arXiv  Detail & Related papers  (2024-10-09T11:46:32Z)
• Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding [125.05295513481035]
  We introduce Visual Contrastive Decoding (VCD), a simple and training-free method that contrasts output distributions derived from original and distorted visual inputs. The proposed VCD effectively reduces the over-reliance on statistical bias and unimodal priors, two essential causes of object hallucinations. Our experiments show that VCD, without either additional training or the usage of external tools, significantly mitigates the object hallucination issue across different LVLM families.
  arXiv  Detail & Related papers  (2023-11-28T16:26:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.