ClinHallu: A Benchmark for Diagnosing Stage-Wise Hallucinations in Medical MLLM Reasoning

• URL: http://arxiv.org/abs/2606.14697v1
• Date: Fri, 12 Jun 2026 17:58:38 GMT
• Title: ClinHallu: A Benchmark for Diagnosing Stage-Wise Hallucinations in Medical MLLM Reasoning
• Abstract summary: We introduce ClinHallu, a benchmark for stage-wise hallucination diagnosis in medical MLLM reasoning.<n>ClinHallu contains 7,031 validated instances, where each instance is augmented with a structured reasoning trace into Visual Recognition, Knowledge Recall, and Reasoning Integration.<n>We show that trace-supervised fine-tuning reduces stage-wise hallucinations.
• Score: 37.1442121485284
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Building trustworthy medical multimodal large language models (MLLMs) is critical for reliable clinical decision support. Existing medical hallucination benchmarks mainly focus on data collection, but often ignore where hallucinations originate within the reasoning process. We find that hallucination sources vary across samples: errors may arise from visual misrecognition, incorrect medical knowledge recall, or flawed reasoning integration. To enable source-level hallucination diagnosis, we introduce ClinHallu, a benchmark for stage-wise hallucination diagnosis in medical MLLM reasoning. ClinHallu contains 7,031 validated instances, where each instance is augmented with a structured reasoning trace decomposed into Visual Recognition, Knowledge Recall, and Reasoning Integration. We also use stage-replacement interventions to measure how correcting specific stages affects the final answer. Beyond evaluation, we show that trace-supervised fine-tuning reduces stage-wise hallucinations. ClinHallu provides a fine-grained hallucination testbed for diagnosing and mitigating reasoning failures in medical MLLMs. The benchmark is publicly available at https://github.com/alibaba-damo-academy/ClinHallu.

Related papers

• Seeing Clearly without Training: Mitigating Hallucinations in Multimodal LLMs for Remote Sensing [38.87853049845978]
  Multimodal large language models (MLLMs) suffer from pronounced hallucinations in remote sensing visual question-answering (RS-VQA)<n>We introduce RSHBench, a protocol-based benchmark for fine-grained diagnosis of factual and logical hallucinations.<n>We propose Relative Attention-Driven Actively Reasoning (RADAR) to guide progressive localization and fine-grained local reasoning at test time.
  arXiv  Detail & Related papers  (2026-03-03T08:53:20Z)
• MedHallTune: An Instruction-Tuning Benchmark for Mitigating Medical Hallucination in Vision-Language Models [81.64135119165277]
  hallucinations can jeopardize clinical decision making, potentially harming the diagnosis and treatments.<n>We propose MedHallTune, a large-scale benchmark designed specifically to evaluate and mitigate hallucinations in medical VLMs.<n>We conduct a comprehensive evaluation of current medical and general VLMs using MedHallTune, assessing their performance across key metrics, including clinical accuracy, relevance, detail level, and risk level.
  arXiv  Detail & Related papers  (2025-02-28T06:59:49Z)
• The HalluRAG Dataset: Detecting Closed-Domain Hallucinations in RAG Applications Using an LLM's Internal States [0.5573267589690007]
  We focus on hallucinations involving information not used in training, which we determine by using recency to ensure the information emerged after a cut-off date.<n>This study investigates these hallucinations by detecting them at sentence level using different internal states of various language models.<n>Our results show that IAVs detect hallucinations as effectively as CEVs and reveal that answerable and unanswerable prompts are encoded differently as separate classifiers.
  arXiv  Detail & Related papers  (2024-12-22T15:08:24Z)
• Detecting and Evaluating Medical Hallucinations in Large Vision Language Models [22.30139330566514]
  Large Vision Language Models (LVLMs) are increasingly integral to healthcare applications. LVLMs inherit susceptibility to hallucinations-a significant concern in high-stakes medical contexts. We introduce Med-HallMark, the first benchmark specifically designed for hallucination detection and evaluation. We also present MediHallDetector, a novel Medical LVLM engineered for precise hallucination detection.
  arXiv  Detail & Related papers  (2024-06-14T17:14:22Z)
• HalluDial: A Large-Scale Benchmark for Automatic Dialogue-Level Hallucination Evaluation [19.318217051269382]
  Large Language Models (LLMs) have significantly advanced the field of Natural Language Processing (NLP) HalluDial is the first comprehensive large-scale benchmark for automatic dialogue-level hallucination evaluation. The benchmark includes 4,094 dialogues with a total of 146,856 samples.
  arXiv  Detail & Related papers  (2024-06-11T08:56:18Z)
• Mitigating Object Hallucination in MLLMs via Data-augmented Phrase-level Alignment [52.43197107069751]
  Multimodal Large Language Models (MLLMs) often generate factually inaccurate information, referred to as hallucination.<n>We introduce Data-augmented Phrase-level Alignment (DPA), a novel loss which can be applied to instruction-tuned off-the-shelf MLLMs to mitigate hallucinations.
  arXiv  Detail & Related papers  (2024-05-28T23:36:00Z)
• DiaHalu: A Dialogue-level Hallucination Evaluation Benchmark for Large Language Models [26.289847386286446]
  We propose DiaHalu, the first dialogue-level hallucination evaluation benchmark to our knowledge. We integrate the collected topics into system prompts and facilitate a dialogue between two ChatGPT3.5. We manually modify the contents that do not adhere to human language conventions and then have LLMs re-generate, simulating authentic human-machine interaction scenarios.
  arXiv  Detail & Related papers  (2024-03-01T15:38:55Z)
• The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models [134.6697160940223]
  hallucination poses great challenge to trustworthy and reliable deployment of large language models. Three key questions should be well studied: how to detect hallucinations (detection), why do LLMs hallucinate (source), and what can be done to mitigate them. This work presents a systematic empirical study on LLM hallucination, focused on the the three aspects of hallucination detection, source and mitigation.
  arXiv  Detail & Related papers  (2024-01-06T12:40:45Z)
• HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data [102.56792377624927]
  hallucinations inherent in machine-generated data remain under-explored. We present a novel hallucination detection and elimination framework, HalluciDoctor, based on the cross-checking paradigm. Our method successfully mitigates 44.6% hallucinations relatively and maintains competitive performance compared to LLaVA.
  arXiv  Detail & Related papers  (2023-11-22T04:52:58Z)
• Evaluation and Analysis of Hallucination in Large Vision-Language Models [49.19829480199372]
  Large Vision-Language Models (LVLMs) have recently achieved remarkable success. LVLMs are still plagued by the hallucination problem. Hallucination refers to the information of LVLMs' responses that does not exist in the visual input.
  arXiv  Detail & Related papers  (2023-08-29T08:51:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.