Explainable Transformer Models for Clinical Prediction Tasks on Structured Electronic Health Records

• URL: http://arxiv.org/abs/2608.20315v1
• Date: Thu, 20 Aug 2026 17:54:17 GMT
• Title: Explainable Transformer Models for Clinical Prediction Tasks on Structured Electronic Health Records
• Abstract summary: BERT-LER is a BERT-style model for coded EHR timelines pretrained and fine-tuned from a de-identified EHR dataset of 75 million patients.<n>We benchmark our approach on the public EHRShot benchmark suite and on an asthma severity progression study based on real-world data.<n>Our architecture and explainability approach can be applied to many therapeutic areas and prediction tasks using language models trained on structured EHRs.
• Score: 0.056212519098516295
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Predictive models over structured electronic health records (EHRs) remain central to machine learning for healthcare, but few have jointly emphasized quantitative laboratory information and interpretability with respect to input medical events. We present BERT-LER, a BERT-style model for coded EHR timelines pretrained and fine-tuned from a de-identified EHR dataset of 75 million patients, that encodes laboratory test results as discrete tokens while retaining graded information through percentile-based binning, paired with Integrated Gradients for token-level attributions grounded in the input EHR sequence. We benchmark our approach on the public EHRShot benchmark suite and on an asthma severity progression study based on real-world data. This addresses a methodological gap in EHR foundation-style modeling by unifying laboratory value representation and explainability in a single framework, while assessing whether both predictive performance and explanations generalize beyond standard clinical prediction tasks. Across EHRShot and asthma tasks, BERT-LER achieves predictive performance that is competitive with, and on laboratory-related tasks often exceeds, publicly available benchmark models, and provides attributions that align with clinically known risk factors. Our architecture and explainability approach can be applied to many therapeutic areas and prediction tasks using language models trained on structured EHRs.

Related papers

• ChatHealthAI: Aligning Electronic Health Record Representations with Large Language Models for Grounded Clinical Reasoning [13.970166588273946]
  Large language models (LLMs) exhibit strong natural-language reasoning abilities for clinical decision support.<n>We propose ChatHealthAI, a multimodal reasoning framework that aligns structured EHR representations with the semantic space of a frozen LLM.<n>Results show that ChatHealthAI improves reasoning quality and interpretability while preserving competitive predictive performance.
  arXiv  Detail & Related papers  (2026-06-01T19:21:18Z)
• EHR-RAGp: Retrieval-Augmented Prototype-Guided Foundation Model for Electronic Health Records [0.9989630065856642]
  EHR contain rich longitudinal patient information and are widely used in predictive modeling applications.<n>We introduce EHR-RAGp, a retrieval-augmented foundation model that dynamically integrates the most relevant patient history across diverse clinical event types.
  arXiv  Detail & Related papers  (2026-05-12T16:17:03Z)
• ReMedi: Reasoner for Medical Clinical Prediction [70.84466325266068]
  We present ReMedi, a framework for improving clinical outcome prediction from EHR.<n>ReMedi generates rationale-answer pairs using a challenging sample regeneration mechanism for complex clinical questions.<n> Experiments on multiple EHR prediction tasks demonstrate substantial gains of up to 19.9 percent over state-of-the-art baselines in terms of F1 score.
  arXiv  Detail & Related papers  (2026-05-02T14:44:49Z)
• Integrating Genomics into Multimodal EHR Foundation Models [56.31910745104141]
  This paper introduces an innovative EHR foundation model that integrates Polygenic Risk Scores (PRS) as a foundational data modality.<n>The framework aims to learn complex relationships between clinical data and genetic predispositions.<n>This approach is pivotal for unlocking new insights into disease prediction, proactive health management, risk stratification, and personalized treatment strategies.
  arXiv  Detail & Related papers  (2025-10-24T15:56:40Z)
• Time-Aware Attention for Enhanced Electronic Health Records Modeling [8.4225455796455]
  TALE-EHR is a Transformer-based framework featuring a novel time-aware attention mechanism that explicitly models continuous temporal gaps.<n>Our approach outperforms state-of-the-art baselines on tasks such as disease progression forecasting.
  arXiv  Detail & Related papers  (2025-07-20T07:32:41Z)
• Representation Learning of Lab Values via Masked AutoEncoders [2.785172582119726]
  We propose Lab-MAE, a novel transformer-based masked autoencoder framework for imputation of sequential lab values.<n>Lab-MAE achieves equitable performance across demographic groups of patients, advancing fairness in clinical predictions.
  arXiv  Detail & Related papers  (2025-01-05T20:26:49Z)
• Recent Advances in Predictive Modeling with Electronic Health Records [71.19967863320647]
  utilizing EHR data for predictive modeling presents several challenges due to its unique characteristics. Deep learning has demonstrated its superiority in various applications, including healthcare.
  arXiv  Detail & Related papers  (2024-02-02T00:31:01Z)
• MedDiffusion: Boosting Health Risk Prediction via Diffusion-based Data Augmentation [58.93221876843639]
  This paper introduces a novel, end-to-end diffusion-based risk prediction model, named MedDiffusion. It enhances risk prediction performance by creating synthetic patient data during training to enlarge sample space. It discerns hidden relationships between patient visits using a step-wise attention mechanism, enabling the model to automatically retain the most vital information for generating high-quality data.
  arXiv  Detail & Related papers  (2023-10-04T01:36:30Z)
• TREEMENT: Interpretable Patient-Trial Matching via Personalized Dynamic Tree-Based Memory Network [54.332862955411656]
  Clinical trials are critical for drug development but often suffer from expensive and inefficient patient recruitment. In recent years, machine learning models have been proposed for speeding up patient recruitment via automatically matching patients with clinical trials. We introduce a dynamic tree-based memory network model named TREEMENT to provide accurate and interpretable patient trial matching.
  arXiv  Detail & Related papers  (2023-07-19T12:35:09Z)
• SANSformers: Self-Supervised Forecasting in Electronic Health Records with Attention-Free Models [48.07469930813923]
  This work aims to forecast the demand for healthcare services, by predicting the number of patient visits to healthcare facilities. We introduce SANSformer, an attention-free sequential model designed with specific inductive biases to cater for the unique characteristics of EHR data. Our results illuminate the promising potential of tailored attention-free models and self-supervised pretraining in refining healthcare utilization predictions across various patient demographics.
  arXiv  Detail & Related papers  (2021-08-31T08:23:56Z)
• Self-Supervised Graph Learning with Hyperbolic Embedding for Temporal Health Event Prediction [13.24834156675212]
  We propose a hyperbolic embedding method with information flow to pre-train medical code representations in a hierarchical structure. We incorporate these pre-trained representations into a graph neural network to detect disease complications. We present a new hierarchy-enhanced historical prediction proxy task in our self-supervised learning framework to fully utilize EHR data.
  arXiv  Detail & Related papers  (2021-06-09T00:42:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.