PaReGTA: An LLM-based EHR Data Encoding Approach to Capture Temporal Information

• URL: http://arxiv.org/abs/2602.19661v2
• Date: Sun, 01 Mar 2026 13:04:03 GMT
• Title: PaReGTA: An LLM-based EHR Data Encoding Approach to Capture Temporal Information
• Abstract summary: PaReGTA converts longitudinal EHR events into visit-level templated text with explicit temporal cues.<n>It learns domain-adapted visit embeddings via lightweight contrastive fine-tuning of a sentence-embedding model.<n>It aggregates visit embeddings into a fixed-dimensional patient representation using hybrid temporal pooling.
• Score: 4.768886949096516
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Temporal information in structured electronic health records (EHRs) is often lost in sparse one-hot or count-based representations, while sequence models can be costly and data-hungry. We propose PaReGTA, an LLM-based encoding framework that (i) converts longitudinal EHR events into visit-level templated text with explicit temporal cues, (ii) learns domain-adapted visit embeddings via lightweight contrastive fine-tuning of a sentence-embedding model, and (iii) aggregates visit embeddings into a fixed-dimensional patient representation using hybrid temporal pooling that captures both recency and globally informative visits. Because PaReGTA does not require training from scratch but instead utilizes a pre-trained LLM, it can perform well even in data-limited cohorts. Furthermore, PaReGTA is model-agnostic and can benefit from future EHR-specialized sentence-embedding models. For interpretability, we introduce PaReGTA-RSS (Representation Shift Score), which quantifies clinically defined factor importance by recomputing representations after targeted factor removal and projecting representation shifts through a machine learning model. On 39,088 migraine patients from the All of Us Research Program, PaReGTA outperforms sparse baselines for migraine type classification while deep sequential models were unstable in our cohort.

Related papers

• Improving Patient Subtyping on Longitudinal Data using Representations from Mamba-based Architecture [5.577596968571274]
  We propose a self-supervised Mamba-based model that learns effective EHR representations and enables enhanced patient subtyping.<n>We evaluate the proposed model on public and private real-world EHR datasets to classify the data.<n>Our findings offer valuable insights into subtyping patients based on temporal records from EHR models.
  arXiv  Detail & Related papers  (2026-06-26T21:45:07Z)
• RePrompT: Recurrent Prompt Tuning for Integrating Structured EHR Encoders with Large Language Models [12.004161606345084]
  We introduce RePrompT, a time-aware framework that integrates structured EHR encoders through prompt tuning.<n> Experiments on MIMIC-III and MIMIC-IV demonstrate that RePrompT consistently outperforms both EHR-based and LLM-based baselines.
  arXiv  Detail & Related papers  (2026-04-20T02:20:13Z)
• Can we generate portable representations for clinical time series data using LLMs? [14.97461269508036]
  We study the question of can large language models (LLMs) create portable patient embeddings i.e. representations of patients.<n>We find that our approach is simple, easy to use and competitive with in-distribution with grid imputation.<n>We find that using these portable representations improves few-shot learning and does not increase demographic recoverability of age or sex relative to baselines.
  arXiv  Detail & Related papers  (2026-03-25T06:34:32Z)
• Cross-Representation Benchmarking in Time-Series Electronic Health Records for Clinical Outcome Prediction [44.23284500920266]
  This benchmark standardises data curation and evaluation across two distinct clinical settings.<n>Experiments reveal that event stream models consistently deliver the strongest performance.<n>We find that feature selection strategies must be adapted to the clinical setting.
  arXiv  Detail & Related papers  (2025-10-10T09:03:47Z)
• Analysis of Transferability Estimation Metrics for Surgical Phase Recognition [3.3285108719932555]
  Fine-tuning pre-trained models has become a cornerstone of modern machine learning, allowing practitioners to achieve high performance with limited labeled data.<n>In surgical video analysis, where expert annotations are especially time-consuming and costly, identifying the most suitable pre-trained model for a downstream task is both critical and challenging.<n>We provide the first comprehensive benchmark of three representative metrics, LogME, H-Score, and TransRate, on two diverse datasets.
  arXiv  Detail & Related papers  (2025-08-22T18:05:33Z)
• Time2Lang: Bridging Time-Series Foundation Models and Large Language Models for Health Sensing Beyond Prompting [3.2688127177376227]
  Large language models (LLMs) show promise for health applications when combined with behavioral sensing data.<n>Traditional approaches convert sensor data into text prompts, but this process is prone to errors, computationally expensive, and requires domain expertise.<n>Here, we present Time2Lang, a framework that directly maps TFM outputs to LLM representations without intermediate text conversion.
  arXiv  Detail & Related papers  (2025-02-11T14:58:54Z)
• CAAT-EHR: Cross-Attentional Autoregressive Transformer for Multimodal Electronic Health Record Embeddings [0.0]
  We introduce CAAT-EHR, a novel architecture designed to generate task-agnostic longitudinal embeddings from raw EHR data.<n>An autoregressive decoder complements the encoder by predicting future time points data during pre-training, ensuring that the resulting embeddings maintain temporal consistency and alignment.
  arXiv  Detail & Related papers  (2025-01-31T05:00:02Z)
• Synthesizing Multimodal Electronic Health Records via Predictive Diffusion Models [69.06149482021071]
  We propose a novel EHR data generation model called EHRPD. It is a diffusion-based model designed to predict the next visit based on the current one while also incorporating time interval estimation. We conduct experiments on two public datasets and evaluate EHRPD from fidelity, privacy, and utility perspectives.
  arXiv  Detail & Related papers  (2024-06-20T02:20:23Z)
• RT-GAN: Recurrent Temporal GAN for Adding Lightweight Temporal Consistency to Frame-Based Domain Translation Approaches [3.7873597471903944]
  We present a lightweight solution with a tunable temporal parameter, RT-GAN, for adding temporal consistency to individual frame-based approaches.<n>We demonstrate the effectiveness of our approach on two challenging use cases in colonoscopy.
  arXiv  Detail & Related papers  (2023-10-02T03:13:26Z)
• ArSDM: Colonoscopy Images Synthesis with Adaptive Refinement Semantic Diffusion Models [69.9178140563928]
  Colonoscopy analysis is essential for assisting clinical diagnosis and treatment. The scarcity of annotated data limits the effectiveness and generalization of existing methods. We propose an Adaptive Refinement Semantic Diffusion Model (ArSDM) to generate colonoscopy images that benefit the downstream tasks.
  arXiv  Detail & Related papers  (2023-09-03T07:55:46Z)
• Interpretable Medical Diagnostics with Structured Data Extraction by Large Language Models [59.89454513692417]
  Tabular data is often hidden in text, particularly in medical diagnostic reports. We propose a novel, simple, and effective methodology for extracting structured tabular data from textual medical reports, called TEMED-LLM. We demonstrate that our approach significantly outperforms state-of-the-art text classification models in medical diagnostics.
  arXiv  Detail & Related papers  (2023-06-08T09:12:28Z)
• An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPT [80.33783969507458]
  The 'Impression' section of a radiology report is a critical basis for communication between radiologists and other physicians. Recent studies have achieved promising results in automatic impression generation using large-scale medical text data. These models often require substantial amounts of medical text data and have poor generalization performance.
  arXiv  Detail & Related papers  (2023-04-17T17:13:42Z)
• Imputation-Free Learning from Incomplete Observations [73.15386629370111]
  We introduce the importance of guided gradient descent (IGSGD) method to train inference from inputs containing missing values without imputation. We employ reinforcement learning (RL) to adjust the gradients used to train the models via back-propagation. Our imputation-free predictions outperform the traditional two-step imputation-based predictions using state-of-the-art imputation methods.
  arXiv  Detail & Related papers  (2021-07-05T12:44:39Z)
• DeepRite: Deep Recurrent Inverse TreatmEnt Weighting for Adjusting Time-varying Confounding in Modern Longitudinal Observational Data [68.29870617697532]
  We propose Deep Recurrent Inverse TreatmEnt weighting (DeepRite) for time-varying confounding in longitudinal data. DeepRite is shown to recover the ground truth from synthetic data, and estimate unbiased treatment effects from real data.
  arXiv  Detail & Related papers  (2020-10-28T15:05:08Z)
• Longitudinal Variational Autoencoder [1.4680035572775534]
  A common approach to analyse high-dimensional data that contains missing values is to learn a low-dimensional representation using variational autoencoders (VAEs) Standard VAEs assume that the learnt representations are i.i.d., and fail to capture the correlations between the data samples. We propose the Longitudinal VAE (L-VAE), that uses a multi-output additive Gaussian process (GP) prior to extend the VAE's capability to learn structured low-dimensional representations. Our approach can simultaneously accommodate both time-varying shared and random effects, produce structured low-dimensional representations
  arXiv  Detail & Related papers  (2020-06-17T10:30:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.