MANGO: Multimodal Acuity traNsformer for intelliGent ICU Outcomes

• URL: http://arxiv.org/abs/2412.17832v1
• Date: Fri, 13 Dec 2024 23:51:15 GMT
• Title: MANGO: Multimodal Acuity traNsformer for intelliGent ICU Outcomes
• Authors: Jiaqing Zhang, Miguel Contreras, Sabyasachi Bandyopadhyay, Andrea Davidson, Jessica Sena, Yuanfang Ren, Ziyuan Guan, Tezcan Ozrazgat-Baslanti, Tyler J. Loftus, Subhash Nerella, Azra Bihorac, Parisa Rashidi,
• Abstract summary: We present MANGO: the Multimodal Acuity traNsformer for intelliGent ICU outcomes.<n>It is designed to enhance the prediction of patient acuity states, transitions, and the need for life-sustaining therapy.
• Score: 11.385654412265461
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Estimation of patient acuity in the Intensive Care Unit (ICU) is vital to ensure timely and appropriate interventions. Advances in artificial intelligence (AI) technologies have significantly improved the accuracy of acuity predictions. However, prior studies using machine learning for acuity prediction have predominantly relied on electronic health records (EHR) data, often overlooking other critical aspects of ICU stay, such as patient mobility, environmental factors, and facial cues indicating pain or agitation. To address this gap, we present MANGO: the Multimodal Acuity traNsformer for intelliGent ICU Outcomes, designed to enhance the prediction of patient acuity states, transitions, and the need for life-sustaining therapy. We collected a multimodal dataset ICU-Multimodal, incorporating four key modalities, EHR data, wearable sensor data, video of patient's facial cues, and ambient sensor data, which we utilized to train MANGO. The MANGO model employs a multimodal feature fusion network powered by Transformer masked self-attention method, enabling it to capture and learn complex interactions across these diverse data modalities even when some modalities are absent. Our results demonstrated that integrating multiple modalities significantly improved the model's ability to predict acuity status, transitions, and the need for life-sustaining therapy. The best-performing models achieved an area under the receiver operating characteristic curve (AUROC) of 0.76 (95% CI: 0.72-0.79) for predicting transitions in acuity status and the need for life-sustaining therapy, while 0.82 (95% CI: 0.69-0.89) for acuity status prediction...

Related papers

• A Foundation Model for Patient Behavior Monitoring and Suicide Detection [42.238354985465975]
  Foundation models (FMs) have achieved remarkable success across various domains, yet their adoption in healthcare remains limited. This paper introduces a novel FM based on a modified vector quantized variational autoencoder (VQ-VAE), specifically designed to process real-world data from wearable devices. We demonstrate that our pretrained FM, trained on a broad cohort of psychiatric patients, performs downstream tasks via its latent representation without fine-tuning on a held-out cohort of suicidal patients.
  arXiv  Detail & Related papers  (2025-03-19T14:01:16Z)
• MELON: Multimodal Mixture-of-Experts with Spectral-Temporal Fusion for Long-Term Mobility Estimation in Critical Care [1.5237145555729716]
  We introduce MELON, a novel framework designed to predict 12-hour mobility status in the critical care setting. We trained and evaluated the MELON model on the multimodal dataset of 126 patients recruited from nine Intensive Care Units at the University of Florida Health Shands Hospital main campus in Gainesville, Florida. Results showed that MELON outperforms conventional approaches for 12-hour mobility status estimation.
  arXiv  Detail & Related papers  (2025-03-10T19:47:46Z)
• Continually Evolved Multimodal Foundation Models for Cancer Prognosis [50.43145292874533]
  Cancer prognosis is a critical task that involves predicting patient outcomes and survival rates. Previous studies have integrated diverse data modalities, such as clinical notes, medical images, and genomic data, leveraging their complementary information. Existing approaches face two major limitations. First, they struggle to incorporate newly arrived data with varying distributions into training, such as patient records from different hospitals. Second, most multimodal integration methods rely on simplistic concatenation or task-specific pipelines, which fail to capture the complex interdependencies across modalities.
  arXiv  Detail & Related papers  (2025-01-30T06:49:57Z)
• Self-Supervised Pre-training Tasks for an fMRI Time-series Transformer in Autism Detection [3.665816629105171]
  Autism Spectrum Disorder (ASD) is a neurodevelopmental condition that encompasses a wide variety of symptoms and degrees of impairment. We have developed a transformer-based self-supervised framework that directly analyzes time-series fMRI data without computing functional connectivity. We show that randomly masking entire ROIs gives better model performance than randomly masking time points in the pre-training step.
  arXiv  Detail & Related papers  (2024-09-18T20:29:23Z)
• Machine Learning for ALSFRS-R Score Prediction: Making Sense of the Sensor Data [44.99833362998488]
  Amyotrophic Lateral Sclerosis (ALS) is a rapidly progressive neurodegenerative disease that presents individuals with limited treatment options. The present investigation, spearheaded by the iDPP@CLEF 2024 challenge, focuses on utilizing sensor-derived data obtained through an app.
  arXiv  Detail & Related papers  (2024-07-10T19:17:23Z)
• BioFusionNet: Deep Learning-Based Survival Risk Stratification in ER+ Breast Cancer Through Multifeature and Multimodal Data Fusion [16.83901927767791]
  We present BioFusionNet, a deep learning framework that fuses image-derived features with genetic and clinical data to obtain a holistic profile. Our model achieves a mean concordance index of 0.77 and a time-dependent area under the curve of 0.84, outperforming state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-02-16T14:19:33Z)
• Detecting Visual Cues in the Intensive Care Unit and Association with Patient Clinical Status [0.9867627975175174]
  Existing patient assessments in the ICU are mostly sporadic and administered manually. We developed a new "masked loss computation" technique that addresses the data imbalance problem. We performed AU inference on 634,054 frames to evaluate the association between facial AUs and clinically important patient conditions.
  arXiv  Detail & Related papers  (2023-11-01T15:07:03Z)
• TMSS: An End-to-End Transformer-based Multimodal Network for Segmentation and Survival Prediction [0.0]
  oncologists do not do this in their analysis but rather fuse the information in their brain from multiple sources such as medical images and patient history. This work proposes a deep learning method that mimics oncologists' analytical behavior when quantifying cancer and estimating patient survival.
  arXiv  Detail & Related papers  (2022-09-12T06:22:05Z)
• Robust and Efficient Medical Imaging with Self-Supervision [80.62711706785834]
  We present REMEDIS, a unified representation learning strategy to improve robustness and data-efficiency of medical imaging AI. We study a diverse range of medical imaging tasks and simulate three realistic application scenarios using retrospective data.
  arXiv  Detail & Related papers  (2022-05-19T17:34:18Z)
• Unsupervised Pre-Training on Patient Population Graphs for Patient-Level Predictions [48.02011627390706]
  Pre-training has shown success in different areas of machine learning, such as Computer Vision (CV), Natural Language Processing (NLP) and medical imaging. In this paper, we apply unsupervised pre-training to heterogeneous, multi-modal EHR data for patient outcome prediction. We find that our proposed graph based pre-training method helps in modeling the data at a population level.
  arXiv  Detail & Related papers  (2022-03-23T17:59:45Z)
• Bridging the Gap Between Patient-specific and Patient-independent Seizure Prediction via Knowledge Distillation [7.2666838978096875]
  Existing approaches typically train models in a patient-specific fashion due to the highly personalized characteristics of epileptic signals. A patient-specific model can then be obtained with the help of distilled knowledge and additional personalized data. Five state-of-the-art seizure prediction methods are trained on the CHB-MIT sEEG database with our proposed scheme.
  arXiv  Detail & Related papers  (2022-02-25T10:30:29Z)
• SANSformers: Self-Supervised Forecasting in Electronic Health Records with Attention-Free Models [48.07469930813923]
  This work aims to forecast the demand for healthcare services, by predicting the number of patient visits to healthcare facilities. We introduce SANSformer, an attention-free sequential model designed with specific inductive biases to cater for the unique characteristics of EHR data. Our results illuminate the promising potential of tailored attention-free models and self-supervised pretraining in refining healthcare utilization predictions across various patient demographics.
  arXiv  Detail & Related papers  (2021-08-31T08:23:56Z)
• Multimodal Gait Recognition for Neurodegenerative Diseases [38.06704951209703]
  We propose a novel hybrid model to learn the gait differences between three neurodegenerative diseases. A new correlative memory neural network architecture is designed for extracting temporal features. Compared with several state-of-the-art techniques, our proposed framework shows more accurate classification results.
  arXiv  Detail & Related papers  (2021-01-07T10:17:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.