PULSE-ICU: A Pretrained Unified Long-Sequence Encoder for Multi-task Prediction in Intensive Care Units

• URL: http://arxiv.org/abs/2511.22199v1
• Date: Thu, 27 Nov 2025 08:10:52 GMT
• Title: PULSE-ICU: A Pretrained Unified Long-Sequence Encoder for Multi-task Prediction in Intensive Care Units
• Authors: Sejeong Jang, Joo Heung Yoon, Hyo Kyung Lee,
• Abstract summary: We present PULSE-ICU, a self-supervised foundation model that learns event-level ICU representations from large-scale EHR sequences.<n>A unified embedding module encodes event identity, continuous values, units, and temporal attributes, while a Longformer-based encoder enables efficient modeling of long trajectories.
• Score: 0.3277163122167433
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Intensive care unit (ICU) data are highly irregular, heterogeneous, and temporally fragmented, posing challenges for generalizable clinical prediction. We present PULSE-ICU, a self-supervised foundation model that learns event-level ICU representations from large-scale EHR sequences without resampling or manual feature engineering. A unified embedding module encodes event identity, continuous values, units, and temporal attributes, while a Longformer-based encoder enables efficient modeling of long trajectories. PULSE-ICU was fine-tuned across 18 prediction tasks, including mortality, intervention forecasting, and phenotype identification, achieving strong performance across task types. External validation on eICU, HiRID, and P12 showed substantial improvements with minimal fine-tuning, demonstrating robustness to domain shift and variable constraints. These findings suggest that foundation-style modeling can improve data efficiency and adaptability, providing a scalable framework for ICU decision support across diverse clinical environments.

Related papers

• DT-ICU: Towards Explainable Digital Twins for ICU Patient Monitoring via Multi-Modal and Multi-Task Iterative Inference [3.630848302035617]
  We introduce DT-ICU, a digital twin framework for continuous risk estimation in intensive care.<n> DT-ICU integrates variable-length clinical time series with static patient information in a unified architecture.<n>We evaluate DT-ICU on the large, publicly available MIMIC-IV dataset, where it consistently outperforms established baseline models.
  arXiv  Detail & Related papers  (2026-01-12T17:54:19Z)
• Adaptive Test-Time Training for Predicting Need for Invasive Mechanical Ventilation in Multi-Center Cohorts [4.250825424649631]
  Test-Time Training (TTT) has emerged as a promising approach to mitigate such shifts.<n>We introduce Adaptive Test-Time Training (AdaTTT), an enhanced TTT framework tailored for EHR-based IMV prediction in ICU settings.
  arXiv  Detail & Related papers  (2025-12-07T04:27:40Z)
• UniSegDiff: Boosting Unified Lesion Segmentation via a Staged Diffusion Model [53.34835793648352]
  We propose UniSegDiff, a novel diffusion model framework for lesion segmentation.<n>UniSegDiff addresses lesion segmentation in a unified manner across multiple modalities and organs.<n> Comprehensive experimental results demonstrate that UniSegDiff significantly outperforms previous state-of-the-art (SOTA) approaches.
  arXiv  Detail & Related papers  (2025-07-24T12:33:10Z)
• CU-ICU: Customizing Unsupervised Instruction-Finetuned Language Models for ICU Datasets via Text-to-Text Transfer Transformer [0.0]
  We introduce CU-ICU, a method for customizing unsupervised instruction-finetuned language models for ICU datasets.<n> CU-ICU employs a sparse fine-tuning approach that combines few-shot prompting with selective parameter updates.<n>We demonstrate that CU-ICU consistently improves predictive accuracy and interpretability over standard fine-tuning methods.
  arXiv  Detail & Related papers  (2025-07-18T04:49:41Z)
• Surgical Foundation Model Leveraging Compression and Entropy Maximization for Image-Guided Surgical Assistance [50.486523249499115]
  Real-time video understanding is critical to guide procedures in minimally invasive surgery (MIS)<n>We propose Compress-to-Explore (C2E), a novel self-supervised framework to learn compact, informative representations from surgical videos.<n>C2E uses entropy-maximizing decoders to compress images while preserving clinically relevant details, improving encoder performance without labeled data.
  arXiv  Detail & Related papers  (2025-05-16T14:02:24Z)
• Improving Representation Learning of Complex Critical Care Data with ICU-BERT [7.287023190850672]
  ICU-BERT is a transformer-based model pre-trained on the MIMIC-IV database.<n>It learns robust representations of complex ICU data with minimal preprocessing.<n>It either compares to or surpasses current performance benchmarks by leveraging fine-tuning.
  arXiv  Detail & Related papers  (2025-02-26T22:16:58Z)
• Scalable Drift Monitoring in Medical Imaging AI [37.1899538374058]
  We develop MMC+, an enhanced framework for scalable drift monitoring. It builds upon the CheXstray framework that introduced real-time drift detection for medical imaging AI models. MMC+ offers a reliable and cost-effective alternative to continuous performance monitoring.
  arXiv  Detail & Related papers  (2024-10-17T02:57:35Z)
• PMT: Progressive Mean Teacher via Exploring Temporal Consistency for Semi-Supervised Medical Image Segmentation [51.509573838103854]
  We propose a semi-supervised learning framework, termed Progressive Mean Teachers (PMT), for medical image segmentation. Our PMT generates high-fidelity pseudo labels by learning robust and diverse features in the training process. Experimental results on two datasets with different modalities, i.e., CT and MRI, demonstrate that our method outperforms the state-of-the-art medical image segmentation approaches.
  arXiv  Detail & Related papers  (2024-09-08T15:02:25Z)
• ASPS: Augmented Segment Anything Model for Polyp Segmentation [77.25557224490075]
  The Segment Anything Model (SAM) has introduced unprecedented potential for polyp segmentation. SAM's Transformer-based structure prioritizes global and low-frequency information. CFA integrates a trainable CNN encoder branch with a frozen ViT encoder, enabling the integration of domain-specific knowledge.
  arXiv  Detail & Related papers  (2024-06-30T14:55:32Z)
• SANSformers: Self-Supervised Forecasting in Electronic Health Records with Attention-Free Models [48.07469930813923]
  This work aims to forecast the demand for healthcare services, by predicting the number of patient visits to healthcare facilities. We introduce SANSformer, an attention-free sequential model designed with specific inductive biases to cater for the unique characteristics of EHR data. Our results illuminate the promising potential of tailored attention-free models and self-supervised pretraining in refining healthcare utilization predictions across various patient demographics.
  arXiv  Detail & Related papers  (2021-08-31T08:23:56Z)
• Temporal Pointwise Convolutional Networks for Length of Stay Prediction in the Intensive Care Unit [0.0]
  We propose a new deep learning model based on the combination of temporal convolution and pointwise (1x1) convolution. It is specifically designed to mitigate common challenges with Electronic Health Records, such as skewness, irregular sampling and missing data. By adding prediction as a side-task, we can improve performance further still, resulting in a mean absolute deviation of 1.55 days (eICU) and 2.28 days (MIMIC-IV) on predicting remaining length of stay.
  arXiv  Detail & Related papers  (2020-07-18T17:30:10Z)
• Predicting Length of Stay in the Intensive Care Unit with Temporal Pointwise Convolutional Networks [0.0]
  We propose a new deep learning model based on the combination of temporal convolution and pointwise (1x1) convolution. It is specifically designed to mitigate for common challenges with Electronic Health Records, such as skewness, irregular sampling and missing data.
  arXiv  Detail & Related papers  (2020-06-29T15:18:33Z)
• Self-Training with Improved Regularization for Sample-Efficient Chest X-Ray Classification [80.00316465793702]
  We present a deep learning framework that enables robust modeling in challenging scenarios. Our results show that using 85% lesser labeled data, we can build predictive models that match the performance of classifiers trained in a large-scale data setting.
  arXiv  Detail & Related papers  (2020-05-03T02:36:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.