Analysis of heart failure patient trajectories using sequence modeling

• URL: http://arxiv.org/abs/2511.16839v2
• Date: Mon, 24 Nov 2025 08:46:39 GMT
• Title: Analysis of heart failure patient trajectories using sequence modeling
• Authors: Falk Dippel, Yinan Yu, Annika Rosengren, Martin Lindgren, Christina E. Lundberg, Erik Aerts, Martin Adiels, Helen Sjöland,
• Abstract summary: Mamba architecture outperformed an advanced Transformer (Transformer++) based on Llama in handling long context lengths.<n>Llama achieves the highest predictive discrimination, best calibration, and showed robustness across all tasks, followed by Mambas.<n>This paper presents a first ablation study with systematic design choices for input tokenization, model configuration and temporal data preprocessing.
• Score: 2.5101181751229915
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Transformers have defined the state-of-the-art for clinical prediction tasks involving electronic health records (EHRs). The recently introduced Mamba architecture outperformed an advanced Transformer (Transformer++) based on Llama in handling long context lengths, while using fewer model parameters. Despite the impressive performance of these architectures, a systematic approach to empirically analyze model performance and efficiency under various settings is not well established in the medical domain. The performances of six sequence models were investigated across three architecture classes (Transformers, Transformers++, Mambas) in a large Swedish heart failure (HF) cohort (N = 42820), providing a clinically relevant case study. Patient data included diagnoses, vital signs, laboratories, medications and procedures extracted from in-hospital EHRs. The models were evaluated on three one-year prediction tasks: clinical instability (a readmission phenotype) after initial HF hospitalization, mortality after initial HF hospitalization and mortality after latest hospitalization. Ablations account for modifications of the EHR-based input patient sequence, architectural model configurations, and temporal preprocessing techniques for data collection. Llama achieves the highest predictive discrimination, best calibration, and showed robustness across all tasks, followed by Mambas. Both architectures demonstrate efficient representation learning, with tiny configurations surpassing other large-scaled Transformers. At equal model size, Llama and Mambas achieve superior performance using 25% less training data. This paper presents a first ablation study with systematic design choices for input tokenization, model configuration and temporal data preprocessing. Future model development in clinical prediction tasks using EHRs could build upon this study's recommendation as a starting point.

Related papers

• The Patient is not a Moving Document: A World Model Training Paradigm for Longitudinal EHR [3.9541492005870604]
  We introduce SMB-Structure, a world model for structured EHR that grounds a joint-embedding prediction architecture (JEPA) with next-token prediction (SFT)<n>We validate across two large-scale cohorts: Memorial Sloan Kettering (23,319 oncology patients; 323,000+ patient-years) and INSPECT (19,402 pulmonary embolism patients)
  arXiv  Detail & Related papers  (2026-01-29T18:49:37Z)
• Investigating the Impact of Histopathological Foundation Models on Regressive Prediction of Homologous Recombination Deficiency [52.50039435394964]
  We systematically evaluate foundation models for regression-based tasks.<n>We extract patch-level features from whole slide images (WSI) using five state-of-the-art foundation models.<n>Models are trained to predict continuous HRD scores based on these extracted features across breast, endometrial, and lung cancer cohorts.
  arXiv  Detail & Related papers  (2026-01-29T14:06:50Z)
• Development and Comparative Analysis of Machine Learning Models for Hypoxemia Severity Triage in CBRNE Emergency Scenarios Using Physiological and Demographic Data from Medical-Grade Devices [0.0]
  Gradient Boosting Models (GBMs) outperformed sequential models in terms of training speed, interpretability, and reliability. A 5-minute prediction window was chosen for timely intervention, with minute-levels standardizing the data. This study highlights ML's potential to improve triage and reduce alarm fatigue.
  arXiv  Detail & Related papers  (2024-10-30T23:24:28Z)
• EHRMamba: Towards Generalizable and Scalable Foundation Models for Electronic Health Records [4.540391547020466]
  We introduce EHRMamba, a robust foundation model built on the Mamba architecture. We introduce a novel approach to Multitask Prompted Finetuning (MPF) for EHR data, which enables EHRMamba to simultaneously learn multiple clinical tasks in a single finetuning phase. Our evaluations on the MIMIC-IV dataset demonstrate that EHRMamba advances state-of-the-art performance across 6 major clinical tasks and excels in EHR forecasting, marking a significant leap forward in the field.
  arXiv  Detail & Related papers  (2024-05-23T13:43:29Z)
• Computational Pathology at Health System Scale -- Self-Supervised Foundation Models from Three Billion Images [30.618749295623363]
  This project aims to train the largest academic foundation model and benchmark the most prominent self-supervised learning algorithms by pre-training. We collected the largest pathology dataset to date, consisting of over 3 billion images from over 423 thousand microscopy slides. Our results demonstrate that pre-training on pathology data is beneficial for downstream performance compared to pre-training on natural images.
  arXiv  Detail & Related papers  (2023-10-10T21:40:19Z)
• The effect of data augmentation and 3D-CNN depth on Alzheimer's Disease detection [51.697248252191265]
  This work summarizes and strictly observes best practices regarding data handling, experimental design, and model evaluation. We focus on Alzheimer's Disease (AD) detection, which serves as a paradigmatic example of challenging problem in healthcare. Within this framework, we train predictive 15 models, considering three different data augmentation strategies and five distinct 3D CNN architectures.
  arXiv  Detail & Related papers  (2023-09-13T10:40:41Z)
• Clinical Deterioration Prediction in Brazilian Hospitals Based on Artificial Neural Networks and Tree Decision Models [56.93322937189087]
  An extremely boosted neural network (XBNet) is used to predict clinical deterioration (CD) The XGBoost model obtained the best results in predicting CD among Brazilian hospitals' data.
  arXiv  Detail & Related papers  (2022-12-17T23:29:14Z)
• Unsupervised Pre-Training on Patient Population Graphs for Patient-Level Predictions [48.02011627390706]
  Pre-training has shown success in different areas of machine learning, such as Computer Vision (CV), Natural Language Processing (NLP) and medical imaging. In this paper, we apply unsupervised pre-training to heterogeneous, multi-modal EHR data for patient outcome prediction. We find that our proposed graph based pre-training method helps in modeling the data at a population level.
  arXiv  Detail & Related papers  (2022-03-23T17:59:45Z)
• SANSformers: Self-Supervised Forecasting in Electronic Health Records with Attention-Free Models [48.07469930813923]
  This work aims to forecast the demand for healthcare services, by predicting the number of patient visits to healthcare facilities. We introduce SANSformer, an attention-free sequential model designed with specific inductive biases to cater for the unique characteristics of EHR data. Our results illuminate the promising potential of tailored attention-free models and self-supervised pretraining in refining healthcare utilization predictions across various patient demographics.
  arXiv  Detail & Related papers  (2021-08-31T08:23:56Z)
• A multi-stage machine learning model on diagnosis of esophageal manometry [50.591267188664666]
  The framework includes deep-learning models at the swallow-level stage and feature-based machine learning models at the study-level stage. This is the first artificial-intelligence-style model to automatically predict CC diagnosis of HRM study from raw multi-swallow data.
  arXiv  Detail & Related papers  (2021-06-25T20:09:23Z)
• Building Deep Learning Models to Predict Mortality in ICU Patients [0.0]
  We propose several deep learning models using the same features as the SAPS II score. Several experiments have been conducted based on the well known clinical dataset Medical Information Mart for Intensive Care III.
  arXiv  Detail & Related papers  (2020-12-11T16:27:04Z)
• Predicting Clinical Diagnosis from Patients Electronic Health Records Using BERT-based Neural Networks [62.9447303059342]
  We show the importance of this problem in medical community. We present a modification of Bidirectional Representations from Transformers (BERT) model for classification sequence. We use a large-scale Russian EHR dataset consisting of about 4 million unique patient visits.
  arXiv  Detail & Related papers  (2020-07-15T09:22:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.