PORTER: Language-Grounded Event Representations for Portable Structured EHR Foundation Models

• URL: http://arxiv.org/abs/2606.24102v1
• Date: Tue, 23 Jun 2026 03:33:25 GMT
• Title: PORTER: Language-Grounded Event Representations for Portable Structured EHR Foundation Models
• Abstract summary: We introduce PORTER, a structured EHR foundation model that decouples event representation from a fixed vocabulary.<n>Porter represents events through their descriptions using a frozen text encoder, integrates numeric values through a dedicated pathway, and learns clinical dynamics over patient timelines.<n>Porter achieved higher AUROC than a task-specific text serialization, at 329-fold lower amortized compute.
• Score: 0.27998963147546146
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Most electronic health record (EHR) foundation models encode clinical events as discrete event tokens from a fixed vocabulary and therefore cannot directly represent events containing unseen concepts or new combinations of concepts and attributes such as numeric values. This limits transfer across institutions and even across deployment pipelines within the same institution. We introduce PORTER, a language-grounded structured EHR foundation model that decouples event representation from this fixed vocabulary. PORTER represents events through their descriptions using a frozen text encoder, integrates numeric values through a dedicated pathway, and learns clinical dynamics over patient timelines with an autoregressively pretrained temporal backbone. Across 74 clinical prediction tasks at a pediatric hospital, PORTER matched the mean AUROC of a fixed-vocabulary model with the same temporal backbone and pretraining objective. When the same patient timelines were rendered using event descriptions not seen during pretraining, PORTER transferred without retraining or vocabulary mapping, recovering 97.1% of the mean AUROC of a model trained directly on the target vocabulary. When transferred to MIMIC, PORTER outperformed the fixed-vocabulary model, which dropped 69% of events because their tokens were unseen. Mechanistic analyses showed cross-vocabulary transfer tracked preservation of patient-level representation geometry rather than the scale of the text encoder, and the numeric pathway improved sensitivity to magnitude without disrupting clinical concept identity. PORTER also achieved higher AUROC than a task-specific text serialization comparator, at 329-fold lower amortized compute. PORTER is a step toward vocabulary-independent EHR foundation models that reduce the need for vocabulary harmonization while preserving in-domain performance and enabling efficient cross-task reuse.

Related papers

• Weave of Formal Thought [51.56484100374058]
  We introduce Weave of Formal Thought (WoFT), a paradigm uniting rigorous syntactic validation with learned structural representations.<n>We present a latent-variable fine-tuning method training the language model to interleave non-terminal grammar symbols directly into generation.<n>For Python, fine-tuning StarCoder2-3B with our RWS objective reduces per-token cross-entropy by 14.3% relative to a text-only SFT baseline.
  arXiv  Detail & Related papers  (2026-06-24T15:58:11Z)
• Something from Nothing: Data Augmentation for Robust Severity Level Estimation of Dysarthric Speech [69.86604856129883]
  Dysarthric speech quality assessment (DSQA) is critical for clinical diagnostics and inclusive speech technologies.<n>We propose a three-stage framework that leverages unlabeled dysarthric speech and large-scale typical speech datasets to scale training.
  arXiv  Detail & Related papers  (2026-03-16T23:00:07Z)
• Tokenization Tradeoffs in Structured EHR Foundation Models [0.23453441553817042]
  Tokenization determines what information is preserved, how efficiently it is encoded, and which relationships must be learned versus precomputed.<n>Here, we pretrained a transformer on pediatric EHR data under a factorial design, varying tokenization along event encoding, time encoding, and workflow annotation.<n>Joint event encoding and positional time encoding outperformed their alternatives while requiring 39.5% and 9.6% fewer pretraining floating-point operations, respectively.
  arXiv  Detail & Related papers  (2026-03-03T22:48:39Z)
• Serialized EHR make for good text representations [1.585843510099207]
  SerialBEHRT is a domain aligned foundation model that extends SciBERT through additional pretraining on structured EHR sequences.<n>We evaluate its effectiveness on the task of antibiotic susceptibility prediction, a clinically meaningful problem in antibiotic stewardship.
  arXiv  Detail & Related papers  (2025-10-11T17:16:15Z)
• Reading Between the Lines: Combining Pause Dynamics and Semantic Coherence for Automated Assessment of Thought Disorder [8.239710313549466]
  This study integrates pause features with semantic coherence metrics across three datasets.<n>Key findings demonstrate that pause features alone robustly predict the severity of formal thought disorder (FTD)<n>These findings suggest that frameworks combining temporal and semantic analyses provide a roadmap for refining the assessment of disorganized speech.
  arXiv  Detail & Related papers  (2025-07-17T22:00:16Z)
• Leveraging large language models and traditional machine learning ensembles for ADHD detection from narrative transcripts [6.55440666066668]
  We introduce an ensemble framework for automatically classifying Attention-Deficit/Hyperactivity Disorder (ADHD) diagnosis (binary) using narrative transcripts.<n>Our approach integrates three complementary models: LLaMA3, RoBERTa, and a Support Vector Machine (SVM)<n> Empirical results show that the ensemble outperforms individual models.
  arXiv  Detail & Related papers  (2025-05-27T15:22:01Z)
• Temporal Entailment Pretraining for Clinical Language Models over EHR Data [9.584923572354045]
  We introduce a novel temporal entailment pretraining objective for language models in the clinical domain.<n>Our method formulates EHR segments as temporally ordered sentence pairs and trains the model to determine whether a later state is entailed by, contradictory to, or neutral with respect to an earlier state.
  arXiv  Detail & Related papers  (2025-04-25T07:30:38Z)
• REBORN: Reinforcement-Learned Boundary Segmentation with Iterative Training for Unsupervised ASR [54.64158282822995]
  We propose REBORN,Reinforcement-Learned Boundary with Iterative Training for Unsupervised ASR. ReBORN alternates between training a segmentation model that predicts the boundaries of the segmental structures in speech signals and training the phoneme prediction model, whose input is the speech feature segmented by the segmentation model, to predict a phoneme transcription. We conduct extensive experiments and find that under the same setting, REBORN outperforms all prior unsupervised ASR models on LibriSpeech, TIMIT, and five non-English languages in Multilingual LibriSpeech.
  arXiv  Detail & Related papers  (2024-02-06T13:26:19Z)
• Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing [53.89917396428747]
  Self-supervised learning in vision-language processing exploits semantic alignment between imaging and text modalities. We explicitly account for prior images and reports when available during both training and fine-tuning. Our approach, named BioViL-T, uses a CNN-Transformer hybrid multi-image encoder trained jointly with a text model.
  arXiv  Detail & Related papers  (2023-01-11T16:35:33Z)
• CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation [113.99145386490639]
  Cross-lingual NER can transfer knowledge between languages via aligned cross-lingual representations or machine translation results. We propose a Cross-lingual Entity Projection framework (CROP) to enable zero-shot cross-lingual NER. We adopt a multilingual labeled sequence translation model to project the tagged sequence back to the target language and label the target raw sentence.
  arXiv  Detail & Related papers  (2022-10-13T13:32:36Z)
• Bridging the Gap between Language Models and Cross-Lingual Sequence Labeling [101.74165219364264]
  Large-scale cross-lingual pre-trained language models (xPLMs) have shown effectiveness in cross-lingual sequence labeling tasks. Despite the great success, we draw an empirical observation that there is a training objective gap between pre-training and fine-tuning stages. In this paper, we first design a pre-training task tailored for xSL named Cross-lingual Language Informative Span Masking (CLISM) to eliminate the objective gap. Second, we present ContrAstive-Consistency Regularization (CACR), which utilizes contrastive learning to encourage the consistency between representations of input parallel
  arXiv  Detail & Related papers  (2022-04-11T15:55:20Z)
• Unsupervised Paraphrasing with Pretrained Language Models [85.03373221588707]
  We propose a training pipeline that enables pre-trained language models to generate high-quality paraphrases in an unsupervised setting. Our recipe consists of task-adaptation, self-supervision, and a novel decoding algorithm named Dynamic Blocking. We show with automatic and human evaluations that our approach achieves state-of-the-art performance on both the Quora Question Pair and the ParaNMT datasets.
  arXiv  Detail & Related papers  (2020-10-24T11:55:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.