Predicting ICU In-Hospital Mortality Using Adaptive Transformer Layer Fusion

• URL: http://arxiv.org/abs/2506.04924v2
• Date: Fri, 06 Jun 2025 05:55:30 GMT
• Title: Predicting ICU In-Hospital Mortality Using Adaptive Transformer Layer Fusion
• Authors: Han Wang, Ruoyun He, Guoguang Lao, Ting Liu, Hejiao Luo, Changqi Qin, Hongying Luo, Junmin Huang, Zihan Wei, Lu Chen, Yongzhi Xu, Ziqian Bi, Junhao Song, Tianyang Wang, Chia Xin Liang, Xinyuan Song, Huafeng Liu, Junfeng Hao, Chunjie Tian,
• Abstract summary: We introduce ALFIA, a modular, attention-based architecture that jointly trains LoRA adapters and an adaptive layer-weighting mechanism to fuse semantic features from a BERT backbone.<n>Trained on our rigorous cw-24 (CriticalWindow-24) benchmark, ALFIA surpasses state-of-the-art classifiers in AUPRC while preserving a balanced precision-recall profile.<n>Our experiments confirm ALFIA's superior early-warning performance, by operating directly on routine clinical text, it furnishes clinicians with a convenient yet robust tool for risk stratification and timely intervention in critical-care settings.
• Score: 12.981138890399242
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Early identification of high-risk ICU patients is crucial for directing limited medical resources. We introduce ALFIA (Adaptive Layer Fusion with Intelligent Attention), a modular, attention-based architecture that jointly trains LoRA (Low-Rank Adaptation) adapters and an adaptive layer-weighting mechanism to fuse multi-layer semantic features from a BERT backbone. Trained on our rigorous cw-24 (CriticalWindow-24) benchmark, ALFIA surpasses state-of-the-art tabular classifiers in AUPRC while preserving a balanced precision-recall profile. The embeddings produced by ALFIA's fusion module, capturing both fine-grained clinical cues and high-level concepts, enable seamless pairing with GBDTs (CatBoost/LightGBM) as ALFIA-boost, and deep neuro networks as ALFIA-nn, yielding additional performance gains. Our experiments confirm ALFIA's superior early-warning performance, by operating directly on routine clinical text, it furnishes clinicians with a convenient yet robust tool for risk stratification and timely intervention in critical-care settings.

Related papers

• NEARL-CLIP: Interacted Query Adaptation with Orthogonal Regularization for Medical Vision-Language Understanding [51.63264715941068]
  textbfNEARL-CLIP (iunderlineNteracted quunderlineEry underlineAdaptation with ounderlineRthogonaunderlineL Regularization) is a novel cross-modality interaction VLM-based framework.
  arXiv  Detail & Related papers  (2025-08-06T05:44:01Z)
• Fitness aligned structural modeling enables scalable virtual screening with AuroBind [56.720030595081845]
  We present AuroBind, a scalable virtual screening framework that fine-tunes a custom atomic-level structural model on million-scale chemogenomic data.<n>AuroBind integrates direct preference optimization, self-distillation from high-confidence complexes, and a teacher-student acceleration strategy.<n>In a prospective screen across ten disease-relevant targets, AuroBind achieved experimental hit rates of 7-69%, with top compounds reaching sub-nanomolar to picomolar potency.
  arXiv  Detail & Related papers  (2025-08-04T07:34:48Z)
• HMSViT: A Hierarchical Masked Self-Supervised Vision Transformer for Corneal Nerve Segmentation and Diabetic Neuropathy Diagnosis [3.8141400767898603]
  Diabetic Peripheral Neuropathy (DPN) affects nearly half of diabetes patients, requiring early detection.<n>We propose HMSViT, a novel Hierarchical Masked Self-Supervised Vision Transformer (HMSViT)<n>HMSViT employs pooling-based hierarchical and dual attention mechanisms with absolute positional encoding, enabling efficient multi-scale feature extraction.<n> Experiments on clinical CCM datasets showed HMSViT achieves state-of-the-art performance, with 61.34% mIoU for nerve segmentation and 70.40% diagnostic accuracy.
  arXiv  Detail & Related papers  (2025-06-24T10:00:23Z)
• PLAME: Leveraging Pretrained Language Models to Generate Enhanced Protein Multiple Sequence Alignments [53.55710514466851]
  Protein structure prediction is essential for drug discovery and understanding biological functions.<n>Most folding models rely heavily on multiple sequence alignments (MSAs) to boost prediction performance.<n>We propose PLAME, a novel MSA design model that leverages evolutionary embeddings from pretrained protein language models.
  arXiv  Detail & Related papers  (2025-06-17T04:11:30Z)
• EKPC: Elastic Knowledge Preservation and Compensation for Class-Incremental Learning [53.88000987041739]
  Class-Incremental Learning (CIL) aims to enable AI models to continuously learn from sequentially arriving data of different classes over time.<n>We propose the Elastic Knowledge Preservation and Compensation (EKPC) method, integrating Importance-aware importance Regularization (IPR) and Trainable Semantic Drift Compensation (TSDC) for CIL.
  arXiv  Detail & Related papers  (2025-06-14T05:19:58Z)
• Channel-Imposed Fusion: A Simple yet Effective Method for Medical Time Series Classification [11.520819583343128]
  This study shifts focus toward a modeling paradigm that emphasizes structural transparency, aligning more closely with the intrinsic characteristics of medical data.<n>We propose a novel method, Channel Imposed Fusion (CIF), which enhances the signal-to-noise ratio through cross-channel information fusion.<n> Experimental results on multiple publicly available EEG and ECG datasets demonstrate that the proposed method not only outperforms existing state-of-the-art (SOTA) approaches in terms of various classification metrics, but also significantly enhances the transparency of the classification process, offering a novel perspective for medical time series classification.
  arXiv  Detail & Related papers  (2025-05-31T01:44:30Z)
• DeepBoost-AF: A Novel Unsupervised Feature Learning and Gradient Boosting Fusion for Robust Atrial Fibrillation Detection in Raw ECG Signals [1.794794261751548]
  Atrial fibrillation (AF) is a prevalent cardiac arrhythmia associated with elevated health risks.<n>This study introduces an innovative hybrid methodology integrating unsupervised deep learning and gradient boosting models to improve AF detection.
  arXiv  Detail & Related papers  (2025-05-30T00:08:56Z)
• Byzantine-Resilient Over-the-Air Federated Learning under Zero-Trust Architecture [68.83934802584899]
  We propose a novel Byzantine-robust FL paradigm for over-the-air transmissions, referred to as federated learning with secure adaptive clustering (FedSAC)<n>FedSAC aims to protect a portion of the devices from attacks through zero trust architecture (ZTA) based Byzantine identification and adaptive device clustering.<n> Numerical results substantiate the superiority of the proposed FedSAC over existing methods in terms of both test accuracy and convergence rate.
  arXiv  Detail & Related papers  (2025-03-24T01:56:30Z)
• Efficient Lung Ultrasound Severity Scoring Using Dedicated Feature Extractor [12.280417624228544]
  MeDiVLAD is a novel pipeline for multi-level lung-ultrasound severity scoring.<n>We leverage self-knowledge distillation to pretrain a vision transformer (ViT) without label and aggregate frame-level features.<n>We show that with minimal finetuning, MeDiVLAD outperforms conventional fully-supervised methods in both frame- and video-level scoring.
  arXiv  Detail & Related papers  (2025-01-21T22:28:22Z)
• ASPS: Augmented Segment Anything Model for Polyp Segmentation [77.25557224490075]
  The Segment Anything Model (SAM) has introduced unprecedented potential for polyp segmentation. SAM's Transformer-based structure prioritizes global and low-frequency information. CFA integrates a trainable CNN encoder branch with a frozen ViT encoder, enabling the integration of domain-specific knowledge.
  arXiv  Detail & Related papers  (2024-06-30T14:55:32Z)
• MSAGPT: Neural Prompting Protein Structure Prediction via MSA Generative Pre-Training [48.398329286769304]
  Multiple Sequence Alignment (MSA) plays a pivotal role in unveiling the evolutionary trajectories of protein families. MSAGPT is a novel approach to prompt protein structure predictions via MSA generative pretraining in the low MSA regime.
  arXiv  Detail & Related papers  (2024-06-08T04:23:57Z)
• Pretraining is All You Need: A Multi-Atlas Enhanced Transformer Framework for Autism Spectrum Disorder Classification [7.790241122137617]
  We propose a novel Multi-Atlas Enhanced Transformer framework, METAFormer, ASD classification. Our framework utilizes resting-state functional magnetic resonance imaging data from the ABIDE I dataset. We show that it surpasses state-of-the-art performance on the ABIDE I dataset, with an average accuracy of 83.7% and an AUC-score of 0.832.
  arXiv  Detail & Related papers  (2023-07-04T15:00:06Z)
• Auto-FedRL: Federated Hyperparameter Optimization for Multi-institutional Medical Image Segmentation [48.821062916381685]
  Federated learning (FL) is a distributed machine learning technique that enables collaborative model training while avoiding explicit data sharing. In this work, we propose an efficient reinforcement learning(RL)-based federated hyperparameter optimization algorithm, termed Auto-FedRL. The effectiveness of the proposed method is validated on a heterogeneous data split of the CIFAR-10 dataset and two real-world medical image segmentation datasets.
  arXiv  Detail & Related papers  (2022-03-12T04:11:42Z)
• Stochastic Optimization of Areas Under Precision-Recall Curves with Provable Convergence [66.83161885378192]
  Area under ROC (AUROC) and precision-recall curves (AUPRC) are common metrics for evaluating classification performance for imbalanced problems. We propose a technical method to optimize AUPRC for deep learning.
  arXiv  Detail & Related papers  (2021-04-18T06:22:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.