Can we generate portable representations for clinical time series data using LLMs?

• URL: http://arxiv.org/abs/2603.23987v1
• Date: Wed, 25 Mar 2026 06:34:32 GMT
• Title: Can we generate portable representations for clinical time series data using LLMs?
• Abstract summary: We study the question of can large language models (LLMs) create portable patient embeddings i.e. representations of patients.<n>We find that our approach is simple, easy to use and competitive with in-distribution with grid imputation.<n>We find that using these portable representations improves few-shot learning and does not increase demographic recoverability of age or sex relative to baselines.
• Score: 14.97461269508036
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deploying clinical ML is slow and brittle: models that work at one hospital often degrade under distribution shifts at the next. In this work, we study a simple question -- can large language models (LLMs) create portable patient embeddings i.e. representations of patients enable a downstream predictor built on one hospital to be used elsewhere with minimal-to-no retraining and fine-tuning. To do so, we map from irregular ICU time series onto concise natural language summaries using a frozen LLM, then embed each summary with a frozen text embedding model to obtain a fixed length vector capable of serving as input to a variety of downstream predictors. Across three cohorts (MIMIC-IV, HIRID, PPICU), on multiple clinically grounded forecasting and classification tasks, we find that our approach is simple, easy to use and competitive with in-distribution with grid imputation, self-supervised representation learning, and time series foundation models, while exhibiting smaller relative performance drops when transferring to new hospitals. We study the variation in performance across prompt design, with structured prompts being crucial to reducing the variance of the predictive models without altering mean accuracy. We find that using these portable representations improves few-shot learning and does not increase demographic recoverability of age or sex relative to baselines, suggesting little additional privacy risk. Our work points to the potential that LLMs hold as tools to enable the scalable deployment of production grade predictive models by reducing the engineering overhead.

Related papers

• LLM4CKD: Large Language Models for Early Stage Chronic Kidney Disease Screening [0.5442955439283729]
  Early screening of chronic kidney disease (CKD) is critical for timely intervention.<n>Most machine learning (ML) and deep learning (DL) approaches require labeled data and model training.<n>This study evaluates the effectiveness of large language models (LLMs) for CKD screening under zero-shot and few-shot in-context learning settings.
  arXiv  Detail & Related papers  (2026-09-03T15:53:20Z)
• LLM4EHR: Aligning Clinical Time Series with Medical Event Sequences via Large Language Models [36.391923477672776]
  Recent research in clinical machine learning has shifted from bespoke supervised models to foundation models.<n>Here, foundation models are pre-trained on mixtures of complex clinical data modalities, useful for various downstream tasks.
  arXiv  Detail & Related papers  (2026-07-16T20:40:04Z)
• PaReGTA: An LLM-based EHR Data Encoding Approach to Capture Temporal Information [4.768886949096516]
  PaReGTA converts longitudinal EHR events into visit-level templated text with explicit temporal cues.<n>It learns domain-adapted visit embeddings via lightweight contrastive fine-tuning of a sentence-embedding model.<n>It aggregates visit embeddings into a fixed-dimensional patient representation using hybrid temporal pooling.
  arXiv  Detail & Related papers  (2026-02-23T10:09:50Z)
• Cross-Representation Benchmarking in Time-Series Electronic Health Records for Clinical Outcome Prediction [44.23284500920266]
  This benchmark standardises data curation and evaluation across two distinct clinical settings.<n>Experiments reveal that event stream models consistently deliver the strongest performance.<n>We find that feature selection strategies must be adapted to the clinical setting.
  arXiv  Detail & Related papers  (2025-10-10T09:03:47Z)
• AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling [53.54623137152208]
  We introduce AutoElicit to extract knowledge from large language models and construct priors for predictive models.<n>We show these priors are informative and can be refined using natural language.<n>We find that AutoElicit yields priors that can substantially reduce error over uninformative priors, using fewer labels, and consistently outperform in-context learning.
  arXiv  Detail & Related papers  (2024-11-26T10:13:39Z)
• TACTiS-2: Better, Faster, Simpler Attentional Copulas for Multivariate Time Series [57.4208255711412]
  Building on copula theory, we propose a simplified objective for the recently-introduced transformer-based attentional copulas (TACTiS) We show that the resulting model has significantly better training dynamics and achieves state-of-the-art performance across diverse real-world forecasting tasks.
  arXiv  Detail & Related papers  (2023-10-02T16:45:19Z)
• Time Associated Meta Learning for Clinical Prediction [78.99422473394029]
  We propose a novel time associated meta learning (TAML) method to make effective predictions at multiple future time points. To address the sparsity problem after task splitting, TAML employs a temporal information sharing strategy to augment the number of positive samples. We demonstrate the effectiveness of TAML on multiple clinical datasets, where it consistently outperforms a range of strong baselines.
  arXiv  Detail & Related papers  (2023-03-05T03:54:54Z)
• CHiLL: Zero-shot Custom Interpretable Feature Extraction from Clinical Notes with Large Language Models [24.048687797399637]
  CHiLL prompts LLMs with expert-crafted queries to generate interpretable features from health records. The resulting noisy labels are then used to train a simple linear classifier. We find that linear models using automatically extracted features are comparably performant to models using reference features, and provide greater interpretability than linear models using "Bag-of-Words" features.
  arXiv  Detail & Related papers  (2023-02-23T21:23:06Z)
• Learning Predictive and Interpretable Timeseries Summaries from ICU Data [33.787187660310444]
  We propose a new procedure to learn summaries of clinical time-series that are both predictive and easily understood by humans. Our learned summaries outperform traditional interpretable model classes and achieve performance comparable to state-of-the-art deep learning models on an in-hospital mortality classification task.
  arXiv  Detail & Related papers  (2021-09-22T21:14:05Z)
• SANSformers: Self-Supervised Forecasting in Electronic Health Records with Attention-Free Models [48.07469930813923]
  This work aims to forecast the demand for healthcare services, by predicting the number of patient visits to healthcare facilities. We introduce SANSformer, an attention-free sequential model designed with specific inductive biases to cater for the unique characteristics of EHR data. Our results illuminate the promising potential of tailored attention-free models and self-supervised pretraining in refining healthcare utilization predictions across various patient demographics.
  arXiv  Detail & Related papers  (2021-08-31T08:23:56Z)
• Towards an Automatic Analysis of CHO-K1 Suspension Growth in Microfluidic Single-cell Cultivation [63.94623495501023]
  We propose a novel Machine Learning architecture, which allows us to infuse a neural deep network with human-powered abstraction on the level of data. Specifically, we train a generative model simultaneously on natural and synthetic data, so that it learns a shared representation, from which a target variable, such as the cell count, can be reliably estimated.
  arXiv  Detail & Related papers  (2020-10-20T08:36:51Z)
• Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype Prediction [55.94378672172967]
  We focus on few-shot disease subtype prediction problem, identifying subgroups of similar patients. We introduce meta learning techniques to develop a new model, which can extract the common experience or knowledge from interrelated clinical tasks. Our new model is built upon a carefully designed meta-learner, called Prototypical Network, that is a simple yet effective meta learning machine for few-shot image classification.
  arXiv  Detail & Related papers  (2020-09-02T02:50:30Z)
• Self-Training with Improved Regularization for Sample-Efficient Chest X-Ray Classification [80.00316465793702]
  We present a deep learning framework that enables robust modeling in challenging scenarios. Our results show that using 85% lesser labeled data, we can build predictive models that match the performance of classifiers trained in a large-scale data setting.
  arXiv  Detail & Related papers  (2020-05-03T02:36:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.