Hallucinations and Key Information Extraction in Medical Texts: A Comprehensive Assessment of Open-Source Large Language Models

• URL: http://arxiv.org/abs/2504.19061v3
• Date: Wed, 20 Aug 2025 14:24:25 GMT
• Title: Hallucinations and Key Information Extraction in Medical Texts: A Comprehensive Assessment of Open-Source Large Language Models
• Authors: Anindya Bijoy Das, Shibbir Ahmed, Shahnewaz Karim Sakib,
• Abstract summary: Large language models (LLMs) have shown significant potential in automating and improving the accuracy of such summarizations.<n>We investigate the effectiveness of open-source LLMs in extracting key events from discharge reports.<n>We also assess the prevalence of various types of hallucinations in the summaries produced by these models.
• Score: 3.1406146587437904
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Clinical summarization is crucial in healthcare as it distills complex medical data into digestible information, enhancing patient understanding and care management. Large language models (LLMs) have shown significant potential in automating and improving the accuracy of such summarizations due to their advanced natural language understanding capabilities. These models are particularly applicable in the context of summarizing medical/clinical texts, where precise and concise information transfer is essential. In this paper, we investigate the effectiveness of open-source LLMs in extracting key events from discharge reports, including admission reasons, major in-hospital events, and critical follow-up actions. In addition, we also assess the prevalence of various types of hallucinations in the summaries produced by these models. Detecting hallucinations is vital as it directly influences the reliability of the information, potentially affecting patient care and treatment outcomes. We conduct comprehensive simulations to rigorously evaluate the performance of these models, further probing the accuracy and fidelity of the extracted content in clinical summarization. Our results reveal that while the LLMs (e.g., Qwen2.5 and DeepSeek-v2) perform quite well in capturing admission reasons and hospitalization events, they are generally less consistent when it comes to identifying follow-up recommendations, highlighting broader challenges in leveraging LLMs for comprehensive summarization.

Related papers

• LLM-Bootstrapped Targeted Finding Guidance for Factual MLLM-based Medical Report Generation [23.74179903717012]
  We introduce Fact-Flow, an innovative framework that separates the process of visual fact identification from the generation of reports.<n>This is achieved by initially predicting clinical findings from the image, which subsequently directs the MLLM to produce a report that is factually precise.<n>A pivotal advancement of our approach is a pipeline that leverages a Large Language Model (LLM) to autonomously create a dataset of labeled medical findings.
  arXiv  Detail & Related papers  (2026-02-28T02:50:20Z)
• Beyond Traditional Diagnostics: Transforming Patient-Side Information into Predictive Insights with Knowledge Graphs and Prototypes [55.310195121276074]
  We propose a Knowledge graph-enhanced, Prototype-aware, and Interpretable (KPI) framework to predict diseases.<n>It integrates structured and trusted medical knowledge into a unified disease knowledge graph, constructs clinically meaningful disease prototypes, and employs contrastive learning to enhance predictive accuracy.<n>It provides clinically valid explanations that closely align with patient narratives, highlighting its practical value for patient-centered healthcare delivery.
  arXiv  Detail & Related papers  (2025-12-09T05:37:54Z)
• Will Large Language Models Transform Clinical Prediction? [3.5700883813789472]
  Large language models (LLMs) are attracting increasing interest in healthcare.<n>Their ability to summarise large datasets effectively, answer questions accurately, and generate synthesised text is widely recognised.
  arXiv  Detail & Related papers  (2025-05-23T17:02:04Z)
• Medical Hallucinations in Foundation Models and Their Impact on Healthcare [53.97060824532454]
  Foundation Models that are capable of processing and generating multi-modal data have transformed AI's role in medicine. We define medical hallucination as any instance in which a model generates misleading medical content. Our results reveal that inference techniques such as Chain-of-Thought (CoT) and Search Augmented Generation can effectively reduce hallucination rates. These findings underscore the ethical and practical imperative for robust detection and mitigation strategies.
  arXiv  Detail & Related papers  (2025-02-26T02:30:44Z)
• Enhancing In-Hospital Mortality Prediction Using Multi-Representational Learning with LLM-Generated Expert Summaries [3.5508427067904864]
  In-hospital mortality (IHM) prediction for ICU patients is critical for timely interventions and efficient resource allocation. This study integrates structured physiological data and clinical notes with Large Language Model (LLM)-generated expert summaries to improve IHM prediction accuracy.
  arXiv  Detail & Related papers  (2024-11-25T16:36:38Z)
• Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrieval [61.70489848327436]
  KARE is a novel framework that integrates knowledge graph (KG) community-level retrieval with large language models (LLMs) reasoning. Extensive experiments demonstrate that KARE outperforms leading models by up to 10.8-15.0% on MIMIC-III and 12.6-12.7% on MIMIC-IV for mortality and readmission predictions.
  arXiv  Detail & Related papers  (2024-10-06T18:46:28Z)
• SemioLLM: Evaluating Large Language Models for Diagnostic Reasoning from Unstructured Clinical Narratives in Epilepsy [45.2233252981348]
  Large Language Models (LLMs) have been shown to encode clinical knowledge.<n>We present SemioLLM, an evaluation framework that benchmarks 6 state-of-the-art models.<n>We show that most LLMs are able to accurately and confidently generate probabilistic predictions of seizure onset zones in the brain.
  arXiv  Detail & Related papers  (2024-07-03T11:02:12Z)
• Mitigating Hallucinations in Large Language Models via Self-Refinement-Enhanced Knowledge Retrieval [14.58181631462891]
  Large language models (LLMs) have demonstrated remarkable capabilities across various domains. Their susceptibility to hallucination poses significant challenges for their deployment in critical areas such as healthcare. We propose Self-Refinement-Enhanced Knowledge Graph Retrieval (Re-KGR) to augment the factuality of LLMs' responses.
  arXiv  Detail & Related papers  (2024-05-10T15:40:50Z)
• VALOR-EVAL: Holistic Coverage and Faithfulness Evaluation of Large Vision-Language Models [57.43276586087863]
  Large Vision-Language Models (LVLMs) suffer from hallucination issues, wherein the models generate plausible-sounding but factually incorrect outputs. Existing benchmarks are often limited in scope, focusing mainly on object hallucinations. We introduce a multi-dimensional benchmark covering objects, attributes, and relations, with challenging images selected based on associative biases.
  arXiv  Detail & Related papers  (2024-04-22T04:49:22Z)
• Clairvoyance: A Pipeline Toolkit for Medical Time Series [95.22483029602921]
  Time-series learning is the bread and butter of data-driven *clinical decision support* Clairvoyance proposes a unified, end-to-end, autoML-friendly pipeline that serves as a software toolkit. Clairvoyance is the first to demonstrate viability of a comprehensive and automatable pipeline for clinical time-series ML.
  arXiv  Detail & Related papers  (2023-10-28T12:08:03Z)
• Self-Verification Improves Few-Shot Clinical Information Extraction [73.6905567014859]
  Large language models (LLMs) have shown the potential to accelerate clinical curation via few-shot in-context learning. They still struggle with issues regarding accuracy and interpretability, especially in mission-critical domains such as health. Here, we explore a general mitigation framework using self-verification, which leverages the LLM to provide provenance for its own extraction and check its own outputs.
  arXiv  Detail & Related papers  (2023-05-30T22:05:11Z)
• SPeC: A Soft Prompt-Based Calibration on Performance Variability of Large Language Model in Clinical Notes Summarization [50.01382938451978]
  We introduce a model-agnostic pipeline that employs soft prompts to diminish variance while preserving the advantages of prompt-based summarization. Experimental findings indicate that our method not only bolsters performance but also effectively curbs variance for various language models.
  arXiv  Detail & Related papers  (2023-03-23T04:47:46Z)
• Benchmarking Heterogeneous Treatment Effect Models through the Lens of Interpretability [82.29775890542967]
  Estimating personalized effects of treatments is a complex, yet pervasive problem. Recent developments in the machine learning literature on heterogeneous treatment effect estimation gave rise to many sophisticated, but opaque, tools. We use post-hoc feature importance methods to identify features that influence the model's predictions.
  arXiv  Detail & Related papers  (2022-06-16T17:59:05Z)
• Rationale production to support clinical decision-making [31.66739991129112]
  We apply InfoCal to the task of predicting hospital readmission using hospital discharge notes. We find each presented model with selected interpretability or feature importance methods yield varying results.
  arXiv  Detail & Related papers  (2021-11-15T09:02:10Z)
• Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration [55.88616573143478]
  Outcome prediction from clinical text can prevent doctors from overlooking possible risks. Diagnoses at discharge, procedures performed, in-hospital mortality and length-of-stay prediction are four common outcome prediction targets. We propose clinical outcome pre-training to integrate knowledge about patient outcomes from multiple public sources.
  arXiv  Detail & Related papers  (2021-02-08T10:26:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.