Fact-Controlled Diagnosis of Hallucinations in Medical Text Summarization

• URL: http://arxiv.org/abs/2506.00448v1
• Date: Sat, 31 May 2025 08:04:37 GMT
• Title: Fact-Controlled Diagnosis of Hallucinations in Medical Text Summarization
• Authors: Suhas BN, Han-Chin Shing, Lei Xu, Mitch Strong, Jon Burnsky, Jessica Ofor, Jordan R. Mason, Susan Chen, Sundararajan Srinivasan, Chaitanya Shivade, Jack Moriarty, Joseph Paul Cohen,
• Abstract summary: Hallucinations in large language models (LLMs) pose significant risks to patient care and clinical decision-making.<n>General-domain detectors struggle to detect clinical hallucinations, and that performance on fact-controlled hallucinations does not reliably predict effectiveness on natural hallucinations.<n>We develop fact-based approaches that count hallucinations, offering explainability not available with existing methods.
• Score: 8.057050705357973
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hallucinations in large language models (LLMs) during summarization of patient-clinician dialogues pose significant risks to patient care and clinical decision-making. However, the phenomenon remains understudied in the clinical domain, with uncertainty surrounding the applicability of general-domain hallucination detectors. The rarity and randomness of hallucinations further complicate their investigation. In this paper, we conduct an evaluation of hallucination detection methods in the medical domain, and construct two datasets for the purpose: A fact-controlled Leave-N-out dataset -- generated by systematically removing facts from source dialogues to induce hallucinated content in summaries; and a natural hallucination dataset -- arising organically during LLM-based medical summarization. We show that general-domain detectors struggle to detect clinical hallucinations, and that performance on fact-controlled hallucinations does not reliably predict effectiveness on natural hallucinations. We then develop fact-based approaches that count hallucinations, offering explainability not available with existing methods. Notably, our LLM-based detectors, which we developed using fact-controlled hallucinations, generalize well to detecting real-world clinical hallucinations. This research contributes a suite of specialized metrics supported by expert-annotated datasets to advance faithful clinical summarization systems.

Related papers

• Theoretical Foundations and Mitigation of Hallucination in Large Language Models [0.0]
  Hallucination in Large Language Models (LLMs) refers to the generation of content that is not faithful to the input or the real-world facts.<n>This paper provides a rigorous treatment of hallucination in LLMs, including formal definitions and theoretical analyses.
  arXiv  Detail & Related papers  (2025-07-20T15:22:34Z)
• Triggering Hallucinations in LLMs: A Quantitative Study of Prompt-Induced Hallucination in Large Language Models [0.0]
  Hallucinations in large language models (LLMs) present a growing challenge across real-world applications.<n>We propose a prompt-based framework to systematically trigger and quantify hallucination.
  arXiv  Detail & Related papers  (2025-05-01T14:33:47Z)
• HalluLens: LLM Hallucination Benchmark [49.170128733508335]
  Large language models (LLMs) often generate responses that deviate from user input or training data, a phenomenon known as "hallucination"<n>This paper introduces a comprehensive hallucination benchmark, incorporating both new extrinsic and existing intrinsic evaluation tasks.
  arXiv  Detail & Related papers  (2025-04-24T13:40:27Z)
• Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations [82.42811602081692]
  This paper introduces a subsequence association framework to systematically trace and understand hallucinations.<n>Key insight is hallucinations that arise when dominant hallucinatory associations outweigh faithful ones.<n>We propose a tracing algorithm that identifies causal subsequences by analyzing hallucination probabilities across randomized input contexts.
  arXiv  Detail & Related papers  (2025-04-17T06:34:45Z)
• Medical Hallucinations in Foundation Models and Their Impact on Healthcare [53.97060824532454]
  Foundation Models that are capable of processing and generating multi-modal data have transformed AI's role in medicine.<n>We define medical hallucination as any instance in which a model generates misleading medical content.<n>Our results reveal that inference techniques such as Chain-of-Thought (CoT) and Search Augmented Generation can effectively reduce hallucination rates.<n>These findings underscore the ethical and practical imperative for robust detection and mitigation strategies.
  arXiv  Detail & Related papers  (2025-02-26T02:30:44Z)
• Can Your Uncertainty Scores Detect Hallucinated Entity? [14.432545893757677]
  We propose a new data set, HalluEntity, which annotates hallucination at the entity level.<n>Based on the dataset, we evaluate uncertainty-based hallucination detection approaches across 17 modern LLMs.<n>Our experimental results show that uncertainty estimation approaches focusing on individual token probabilities tend to over-predict hallucinations.
  arXiv  Detail & Related papers  (2025-02-17T16:01:41Z)
• Valuable Hallucinations: Realizable Non-realistic Propositions [2.451326684641447]
  This paper introduces the first formal definition of valuable hallucinations in large language models (LLMs)<n>We focus on the potential value that certain types of hallucinations can offer in specific contexts.<n>We present experiments using the Qwen2.5 model and HalluQA dataset, employing ReAct prompting to control and optimize hallucinations.
  arXiv  Detail & Related papers  (2025-02-16T12:59:11Z)
• Hal-Eval: A Universal and Fine-grained Hallucination Evaluation Framework for Large Vision Language Models [35.45859414670449]
  We introduce a refined taxonomy of hallucinations, featuring a new category: Event Hallucination. We then utilize advanced LLMs to generate and filter fine grained hallucinatory data consisting of various types of hallucinations. The proposed benchmark distinctively assesses LVLMs ability to tackle a broad spectrum of hallucinations.
  arXiv  Detail & Related papers  (2024-02-24T05:14:52Z)
• Fine-grained Hallucination Detection and Editing for Language Models [109.56911670376932]
  Large language models (LMs) are prone to generate factual errors, which are often called hallucinations. We introduce a comprehensive taxonomy of hallucinations and argue that hallucinations manifest in diverse forms. We propose a novel task of automatic fine-grained hallucination detection and construct a new evaluation benchmark, FavaBench.
  arXiv  Detail & Related papers  (2024-01-12T19:02:48Z)
• HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data [102.56792377624927]
  hallucinations inherent in machine-generated data remain under-explored. We present a novel hallucination detection and elimination framework, HalluciDoctor, based on the cross-checking paradigm. Our method successfully mitigates 44.6% hallucinations relatively and maintains competitive performance compared to LLaVA.
  arXiv  Detail & Related papers  (2023-11-22T04:52:58Z)
• AutoHall: Automated Hallucination Dataset Generation for Large Language Models [56.92068213969036]
  This paper introduces a method for automatically constructing model-specific hallucination datasets based on existing fact-checking datasets called AutoHall. We also propose a zero-resource and black-box hallucination detection method based on self-contradiction.
  arXiv  Detail & Related papers  (2023-09-30T05:20:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.