Temporal Cross-Attention for Dynamic Embedding and Tokenization of Multimodal Electronic Health Records

• URL: http://arxiv.org/abs/2403.04012v2
• Date: Mon, 1 Apr 2024 20:26:01 GMT
• Title: Temporal Cross-Attention for Dynamic Embedding and Tokenization of Multimodal Electronic Health Records
• Authors: Yingbo Ma, Suraj Kolla, Dhruv Kaliraman, Victoria Nolan, Zhenhong Hu, Ziyuan Guan, Yuanfang Ren, Brooke Armfield, Tezcan Ozrazgat-Baslanti, Tyler J. Loftus, Parisa Rashidi, Azra Bihorac, Benjamin Shickel,
• Abstract summary: We introduce a dynamic embedding and tokenization framework for precise representation of multimodal clinical time series. Our framework outperformed baseline approaches on the task of predicting the occurrence of nine postoperative complications.
• Score: 1.6609516435725236
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The breadth, scale, and temporal granularity of modern electronic health records (EHR) systems offers great potential for estimating personalized and contextual patient health trajectories using sequential deep learning. However, learning useful representations of EHR data is challenging due to its high dimensionality, sparsity, multimodality, irregular and variable-specific recording frequency, and timestamp duplication when multiple measurements are recorded simultaneously. Although recent efforts to fuse structured EHR and unstructured clinical notes suggest the potential for more accurate prediction of clinical outcomes, less focus has been placed on EHR embedding approaches that directly address temporal EHR challenges by learning time-aware representations from multimodal patient time series. In this paper, we introduce a dynamic embedding and tokenization framework for precise representation of multimodal clinical time series that combines novel methods for encoding time and sequential position with temporal cross-attention. Our embedding and tokenization framework, when integrated into a multitask transformer classifier with sliding window attention, outperformed baseline approaches on the exemplar task of predicting the occurrence of nine postoperative complications of more than 120,000 major inpatient surgeries using multimodal data from three hospitals and two academic health centers in the United States.

Related papers

• CTPD: Cross-Modal Temporal Pattern Discovery for Enhanced Multimodal Electronic Health Records Analysis [46.56667527672019]
  We introduce a Cross-Modal Temporal Pattern Discovery (CTPD) framework, designed to efficiently extract meaningful cross-modal temporal patterns from multimodal EHR data. Our approach introduces shared initial temporal pattern representations which are refined using slot attention to generate temporal semantic embeddings.
  arXiv  Detail & Related papers  (2024-11-01T15:54:07Z)
• Unlocking the Power of Spatial and Temporal Information in Medical Multimodal Pre-training [99.2891802841936]
  We introduce the Med-ST framework for fine-grained spatial and temporal modeling. For spatial modeling, Med-ST employs the Mixture of View Expert (MoVE) architecture to integrate different visual features from both frontal and lateral views. For temporal modeling, we propose a novel cross-modal bidirectional cycle consistency objective by forward mapping classification (FMC) and reverse mapping regression (RMR)
  arXiv  Detail & Related papers  (2024-05-30T03:15:09Z)
• Global Contrastive Training for Multimodal Electronic Health Records with Language Supervision [1.6245786035158123]
  This paper introduces a novel multimodal contrastive learning framework, specifically focusing on medical time series and clinical notes. The framework integrates temporal cross-attention transformers with a dynamic embedding and tokenization scheme for learning multimodal feature representations. Experiments with a real-world EHR dataset demonstrated that our framework outperformed state-of-the-art approaches on the exemplar task of predicting the occurrence of nine postoperative complications.
  arXiv  Detail & Related papers  (2024-04-10T04:19:59Z)
• ICU Mortality Prediction Using Long Short-Term Memory Networks [0.0]
  We implement an automatic data-driven system, which analyzes large amounts of temporal data derived from Electronic Health Records (EHRs) We extract high-level information so as to predict in-hospital mortality and Length of Stay (LOS) early. Experiments highlight the efficiency of LSTM model with rigorous time-series measurements for building real-world prediction engines.
  arXiv  Detail & Related papers  (2023-08-18T09:44:47Z)
• Time Associated Meta Learning for Clinical Prediction [78.99422473394029]
  We propose a novel time associated meta learning (TAML) method to make effective predictions at multiple future time points. To address the sparsity problem after task splitting, TAML employs a temporal information sharing strategy to augment the number of positive samples. We demonstrate the effectiveness of TAML on multiple clinical datasets, where it consistently outperforms a range of strong baselines.
  arXiv  Detail & Related papers  (2023-03-05T03:54:54Z)
• T-Phenotype: Discovering Phenotypes of Predictive Temporal Patterns in Disease Progression [82.85825388788567]
  We develop a novel temporal clustering method, T-Phenotype, to discover phenotypes of predictive temporal patterns from labeled time-series data. We show that T-Phenotype achieves the best phenotype discovery performance over all the evaluated baselines.
  arXiv  Detail & Related papers  (2023-02-24T13:30:35Z)
• MVMTnet: A Multi-variate Multi-modal Transformer for Multi-class Classification of Cardiac Irregularities Using ECG Waveforms and Clinical Notes [4.648677931378919]
  Deep learning can be used to optimize diagnosis and patient monitoring for clinical-based applications. For cardiovascular disease, one such condition where the rising number of patients increasingly outweighs the availability of medical resources in different parts of the world, a core challenge is the automated classification of various cardiac abnormalities. The proposed novel multi-modal Transformer architecture would be able to accurately perform this task while demonstrating the cross-domain effectiveness of Transformers.
  arXiv  Detail & Related papers  (2023-02-21T21:38:41Z)
• Multi-view Integration Learning for Irregularly-sampled Clinical Time Series [1.9639092030562577]
  We propose a multi-view features integration learning from irregular time series data by self-attention mechanism in an imputation-free manner. We explicitly learn the relationships among the observed values, missing indicators, and time interval between the consecutive observations, simultaneously. We build an attention-based decoder as a missing value imputer that helps empower the representation learning of the inter-relations among multi-view observations.
  arXiv  Detail & Related papers  (2021-01-25T10:02:50Z)
• MIA-Prognosis: A Deep Learning Framework to Predict Therapy Response [58.0291320452122]
  This paper aims at a unified deep learning approach to predict patient prognosis and therapy response. We formalize the prognosis modeling as a multi-modal asynchronous time series classification task. Our predictive model could further stratify low-risk and high-risk patients in terms of long-term survival.
  arXiv  Detail & Related papers  (2020-10-08T15:30:17Z)
• BiteNet: Bidirectional Temporal Encoder Network to Predict Medical Outcomes [53.163089893876645]
  We propose a novel self-attention mechanism that captures the contextual dependency and temporal relationships within a patient's healthcare journey. An end-to-end bidirectional temporal encoder network (BiteNet) then learns representations of the patient's journeys. We have evaluated the effectiveness of our methods on two supervised prediction and two unsupervised clustering tasks with a real-world EHR dataset.
  arXiv  Detail & Related papers  (2020-09-24T00:42:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.