Time-Aware Attention for Enhanced Electronic Health Records Modeling

• URL: http://arxiv.org/abs/2507.14847v1
• Date: Sun, 20 Jul 2025 07:32:41 GMT
• Title: Time-Aware Attention for Enhanced Electronic Health Records Modeling
• Authors: Junhan Yu, Zhunyi Feng, Junwei Lu, Tianxi Cai, Doudou Zhou,
• Abstract summary: TALE-EHR is a Transformer-based framework featuring a novel time-aware attention mechanism that explicitly models continuous temporal gaps.<n>Our approach outperforms state-of-the-art baselines on tasks such as disease progression forecasting.
• Score: 8.4225455796455
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Electronic Health Records (EHR) contain valuable clinical information for predicting patient outcomes and guiding healthcare decisions. However, effectively modeling Electronic Health Records (EHRs) requires addressing data heterogeneity and complex temporal patterns. Standard approaches often struggle with irregular time intervals between clinical events. We propose TALE-EHR, a Transformer-based framework featuring a novel time-aware attention mechanism that explicitly models continuous temporal gaps to capture fine-grained sequence dynamics. To complement this temporal modeling with robust semantics, TALE-EHR leverages embeddings derived from standardized code descriptions using a pre-trained Large Language Model (LLM), providing a strong foundation for understanding clinical concepts. Experiments on the MIMIC-IV and PIC dataset demonstrate that our approach outperforms state-of-the-art baselines on tasks such as disease progression forecasting. TALE-EHR underscores the benefit of integrating explicit, continuous temporal modeling with strong semantic representations provides a powerful solution for advancing EHR analysis.

Related papers

• From EHRs to Patient Pathways: Scalable Modeling of Longitudinal Health Trajectories with LLMs [38.49879425944787]
  We propose a novel approach to patient pathway modeling by transforming diverse electronic health record (EHR) data into a structured representation.<n>We introduce a novel summary mechanism that embeds long-term temporal context into topic-specific summary tokens, improving performance over text-only models.
  arXiv  Detail & Related papers  (2025-06-05T09:54:01Z)
• Temporal Entailment Pretraining for Clinical Language Models over EHR Data [9.584923572354045]
  We introduce a novel temporal entailment pretraining objective for language models in the clinical domain.<n>Our method formulates EHR segments as temporally ordered sentence pairs and trains the model to determine whether a later state is entailed by, contradictory to, or neutral with respect to an earlier state.
  arXiv  Detail & Related papers  (2025-04-25T07:30:38Z)
• Deep State-Space Generative Model For Correlated Time-to-Event Predictions [54.3637600983898]
  We propose a deep latent state-space generative model to capture the interactions among different types of correlated clinical events. Our method also uncovers meaningful insights about the latent correlations among mortality and different types of organ failures.
  arXiv  Detail & Related papers  (2024-07-28T02:42:36Z)
• Synthesizing Multimodal Electronic Health Records via Predictive Diffusion Models [69.06149482021071]
  We propose a novel EHR data generation model called EHRPD. It is a diffusion-based model designed to predict the next visit based on the current one while also incorporating time interval estimation. We conduct experiments on two public datasets and evaluate EHRPD from fidelity, privacy, and utility perspectives.
  arXiv  Detail & Related papers  (2024-06-20T02:20:23Z)
• Recent Advances in Predictive Modeling with Electronic Health Records [71.19967863320647]
  utilizing EHR data for predictive modeling presents several challenges due to its unique characteristics. Deep learning has demonstrated its superiority in various applications, including healthcare.
  arXiv  Detail & Related papers  (2024-02-02T00:31:01Z)
• MedDiffusion: Boosting Health Risk Prediction via Diffusion-based Data Augmentation [58.93221876843639]
  This paper introduces a novel, end-to-end diffusion-based risk prediction model, named MedDiffusion. It enhances risk prediction performance by creating synthetic patient data during training to enlarge sample space. It discerns hidden relationships between patient visits using a step-wise attention mechanism, enabling the model to automatically retain the most vital information for generating high-quality data.
  arXiv  Detail & Related papers  (2023-10-04T01:36:30Z)
• COPER: Continuous Patient State Perceiver [13.735956129637945]
  We propose a novel COntinuous patient state PERceiver model, called COPER, to cope with irregular time-series in EHRs. neural ordinary differential equations (ODEs) help COPER to generate regular time-series to feed to Perceiver model. To evaluate the performance of the proposed model, we use in-hospital mortality prediction task on MIMIC-III dataset.
  arXiv  Detail & Related papers  (2022-08-05T14:32:57Z)
• Cumulative Stay-time Representation for Electronic Health Records in Medical Event Time Prediction [8.261597797345342]
  We propose a novel data representation for EHR called cumulative stay-time representation (CTR) CTR directly models such cumulative health conditions. We derive a trainable construction of CTR based on neural networks that has the flexibility to fit the target data.
  arXiv  Detail & Related papers  (2022-04-28T12:34:41Z)
• SANSformers: Self-Supervised Forecasting in Electronic Health Records with Attention-Free Models [48.07469930813923]
  This work aims to forecast the demand for healthcare services, by predicting the number of patient visits to healthcare facilities. We introduce SANSformer, an attention-free sequential model designed with specific inductive biases to cater for the unique characteristics of EHR data. Our results illuminate the promising potential of tailored attention-free models and self-supervised pretraining in refining healthcare utilization predictions across various patient demographics.
  arXiv  Detail & Related papers  (2021-08-31T08:23:56Z)
• Self-Supervised Graph Learning with Hyperbolic Embedding for Temporal Health Event Prediction [13.24834156675212]
  We propose a hyperbolic embedding method with information flow to pre-train medical code representations in a hierarchical structure. We incorporate these pre-trained representations into a graph neural network to detect disease complications. We present a new hierarchy-enhanced historical prediction proxy task in our self-supervised learning framework to fully utilize EHR data.
  arXiv  Detail & Related papers  (2021-06-09T00:42:44Z)
• MIA-Prognosis: A Deep Learning Framework to Predict Therapy Response [58.0291320452122]
  This paper aims at a unified deep learning approach to predict patient prognosis and therapy response. We formalize the prognosis modeling as a multi-modal asynchronous time series classification task. Our predictive model could further stratify low-risk and high-risk patients in terms of long-term survival.
  arXiv  Detail & Related papers  (2020-10-08T15:30:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.