Is Clinical Text Enough? A Multimodal Study on Mortality Prediction in Heart Failure Patients

• URL: http://arxiv.org/abs/2604.01924v1
• Date: Thu, 02 Apr 2026 11:41:36 GMT
• Title: Is Clinical Text Enough? A Multimodal Study on Mortality Prediction in Heart Failure Patients
• Abstract summary: We evaluate transformer-based models on a French heart failure cohort.<n>We show that enriching clinical text with entity-level representations improves prediction over CLS embeddings alone.<n>Large language models perform inconsistently across modalities and decoding strategies.
• Score: 1.6784199253342376
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate short-term mortality prediction in heart failure (HF) remains challenging, particularly when relying on structured electronic health record (EHR) data alone. We evaluate transformer-based models on a French HF cohort, comparing text-only, structured-only, multimodal, and LLM-based approaches. Our results show that enriching clinical text with entity-level representations improves prediction over CLS embeddings alone, and that supervised multimodal fusion of text and structured variables achieves the best overall performance. In contrast, large language models perform inconsistently across modalities and decoding strategies, with text-only prompts outperforming structured or multimodal inputs. These findings highlight that entity-aware multimodal transformers offer the most reliable solution for short-term HF outcome prediction, while current LLM prompting remains limited for clinical decision support.

Related papers

• Concept-Grounded Reasoning with Prompt-Driven Localization for Interpretable Structured Report Generation [66.62662330558906]
  CORAL is a multimodal framework that integrates spatial grounding and concept-level supervision into a unified reasoning process.<n>CORAL employs a prompt-driven medical segmentation model to localize lesions and predicts multi-class clinical attributes through a Concept Bottleneck module.<n>Experiments on BUS-CoT and IU X-ray datasets demonstrate consistent improvements in diagnostic accuracy, concept consistency, and report quality over strong general-purpose and medical MLLMs.
  arXiv  Detail & Related papers  (2026-09-14T10:21:23Z)
• Domain-Adapted Small Language Models for Reliable Clinical Triage [4.836986736624673]
  This study evaluates whether open-source small language models (SLMs) can serve as reliable, privacy-preserving decision-support tools for clinical triage.
  arXiv  Detail & Related papers  (2026-04-29T15:00:09Z)
• Dynamic Summary Generation for Interpretable Multimodal Depression Detection [23.358673852385337]
  Depression remains widely underdiagnosed and undertreated because stigma and subjective symptom ratings hinder reliable screening.<n>We propose a coarse-to-fine, multi-stage framework that leverages large language models for accurate and interpretable detection.
  arXiv  Detail & Related papers  (2026-04-13T11:34:40Z)
• Cross-Linguistic Persona-Driven Data Synthesis for Robust Multimodal Cognitive Decline Detection [20.599682298329213]
  We introduce SynCog, a novel framework integrating controllable zero-shot multimodal data synthesis with Chain-of-Thought deduction fine-tuning.<n>This generative paradigm enables the rapid, zero-shot expansion of clinical corpora across diverse languages.<n>Experiments on the ADReSS and ADReSSo benchmarks demonstrate that augmenting limited clinical data with synthetic phenotypes yields competitive diagnostic performance.
  arXiv  Detail & Related papers  (2026-02-08T14:10:05Z)
• A Federated and Parameter-Efficient Framework for Large Language Model Training in Medicine [59.78991974851707]
  Large language models (LLMs) have demonstrated strong performance on medical benchmarks, including question answering and diagnosis.<n>Most medical LLMs are trained on data from a single institution, which faces limitations in generalizability and safety in heterogeneous systems.<n>We introduce the model-agnostic and parameter-efficient federated learning framework for adapting LLMs to medical applications.
  arXiv  Detail & Related papers  (2026-01-29T18:48:21Z)
• ClinStructor: AI-Powered Structuring of Unstructured Clinical Texts [3.073796943975155]
  We present ClinStructor, a pipeline that leverages large language models (LLMs) to convert clinical free-text into structured, task-specific question-answer pairs prior to predictive modeling.<n>Our method substantially enhances transparency and controllability and only leads to a modest reduction in predictive performance.
  arXiv  Detail & Related papers  (2025-11-14T21:21:16Z)
• MedAlign: A Synergistic Framework of Multimodal Preference Optimization and Federated Meta-Cognitive Reasoning [52.064286116035134]
  We develop MedAlign, a framework to ensure visually accurate LVLM responses for Medical Visual Question Answering (Med-VQA)<n>We first propose a multimodal Direct Preference Optimization (mDPO) objective to align preference learning with visual context.<n>We then design a Retrieval-Aware Mixture-of-Experts (RA-MoE) architecture that utilizes image and text similarity to route queries to a specialized and context-augmented LVLM.
  arXiv  Detail & Related papers  (2025-10-24T02:11:05Z)
• Analysis of Image-and-Text Uncertainty Propagation in Multimodal Large Language Models with Cardiac MR-Based Applications [10.096013178241117]
  Multimodal large language models (MLLMs) can process and integrate information from multimodality sources, such as text and images.<n> uncertainties due to individual uni-modal data and potential clinical applications are yet fully understood.<n>We propose a multimodal uncertainty propagation model (MUPM) based on uncertainty propagation.
  arXiv  Detail & Related papers  (2025-07-17T09:34:21Z)
• Attribute Structuring Improves LLM-Based Evaluation of Clinical Text Summaries [56.31117605097345]
  Large language models (LLMs) have shown the potential to generate accurate clinical text summaries, but still struggle with issues regarding grounding and evaluation.<n>Here, we explore a general mitigation framework using Attribute Structuring (AS), which structures the summary evaluation process.<n>AS consistently improves the correspondence between human annotations and automated metrics in clinical text summarization.
  arXiv  Detail & Related papers  (2024-03-01T21:59:03Z)
• XAI for In-hospital Mortality Prediction via Multimodal ICU Data [57.73357047856416]
  We propose an efficient, explainable AI solution for predicting in-hospital mortality via multimodal ICU data. We employ multimodal learning in our framework, which can receive heterogeneous inputs from clinical data and make decisions. Our framework can be easily transferred to other clinical tasks, which facilitates the discovery of crucial factors in healthcare research.
  arXiv  Detail & Related papers  (2023-12-29T14:28:04Z)
• Self-Verification Improves Few-Shot Clinical Information Extraction [73.6905567014859]
  Large language models (LLMs) have shown the potential to accelerate clinical curation via few-shot in-context learning. They still struggle with issues regarding accuracy and interpretability, especially in mission-critical domains such as health. Here, we explore a general mitigation framework using self-verification, which leverages the LLM to provide provenance for its own extraction and check its own outputs.
  arXiv  Detail & Related papers  (2023-05-30T22:05:11Z)
• P-Transformer: A Prompt-based Multimodal Transformer Architecture For Medical Tabular Data [2.4688646371447898]
  We propose PTransformer, a underlinePrompt-based multimodal underlineTransformer architecture designed specifically for medical tabular data.<n>The framework efficiently encodes diverse modalities from both structured and unstructured data into a harmonized language semantic space.<n>PTransformer demonstrated the improvements with 10.9%/11.0% on RMSE/MAE, 0.5%/2.2% on RMSE/MAE, and 1.6%/0.8% on BACC/AUROC compared to state-of-the-art (SOTA) baselines in predictability.
  arXiv  Detail & Related papers  (2023-03-30T14:25:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.