A Weakly Supervised Transformer for Rare Disease Diagnosis and Subphenotyping from EHRs with Pulmonary Case Studies

• URL: http://arxiv.org/abs/2507.02998v2
• Date: Thu, 16 Oct 2025 22:43:35 GMT
• Title: A Weakly Supervised Transformer for Rare Disease Diagnosis and Subphenotyping from EHRs with Pulmonary Case Studies
• Authors: Kimberly F. Greco, Zongxin Yang, Mengyan Li, Han Tong, Sara Morini Sweet, Alon Geva, Kenneth D. Mandl, Benjamin A. Raby, Tianxi Cai,
• Abstract summary: We propose WEST (WEakly Supervised Transformer for rare disease phenotyping and subphenotyping from EHRs) to enable large-scale phenotyping.<n>We evaluate WEST on two rare pulmonary diseases using EHR data from Boston Children's Hospital and show that it outperforms existing methods in phenotype classification, identification of clinically meaningful subphenotypes, and prediction of disease progression.
• Score: 28.253741893497136
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Rare diseases affect an estimated 300-400 million people worldwide, yet individual conditions remain underdiagnosed and poorly characterized due to their low prevalence and limited clinician familiarity. Computational phenotyping offers a scalable approach to improving rare disease detection, but algorithm development is hindered by the scarcity of high-quality labeled data for training. Expert-labeled datasets from chart reviews and registries are clinically accurate but limited in scope and availability, whereas labels derived from electronic health records (EHRs) provide broader coverage but are often noisy or incomplete. To address these challenges, we propose WEST (WEakly Supervised Transformer for rare disease phenotyping and subphenotyping from EHRs), a framework that combines routinely collected EHR data with a limited set of expert-validated cases and controls to enable large-scale phenotyping. At its core, WEST employs a weakly supervised transformer model trained on extensive probabilistic silver-standard labels - derived from both structured and unstructured EHR features - that are iteratively refined during training to improve model calibration. We evaluate WEST on two rare pulmonary diseases using EHR data from Boston Children's Hospital and show that it outperforms existing methods in phenotype classification, identification of clinically meaningful subphenotypes, and prediction of disease progression. By reducing reliance on manual annotation, WEST enables data-efficient rare disease phenotyping that improves cohort definition, supports earlier and more accurate diagnosis, and accelerates data-driven discovery for the rare disease community.

Related papers

• Enhancing Phenotype Discovery in Electronic Health Records through Prior Knowledge-Guided Unsupervised Learning [1.0414011156637284]
  We operationalize a Bayesian latent class framework for phenotyping.<n>We identify an asthma sub-phenotype informed by features of Type 2 (T2) inflammation.
  arXiv  Detail & Related papers  (2025-11-03T22:25:07Z)
• PhenoKG: Knowledge Graph-Driven Gene Discovery and Patient Insights from Phenotypes Alone [40.61937241424789]
  We propose a graph-based approach for predicting causative genes from patient phenotypes, with or without an available list of candidate genes.<n>Our model, combining graph neural networks and transformers, achieves substantial improvements over the current state-of-the-art.
  arXiv  Detail & Related papers  (2025-06-16T05:54:12Z)
• Robust Molecular Property Prediction via Densifying Scarce Labeled Data [53.24886143129006]
  In drug discovery, compounds most critical for advancing research often lie beyond the training set.<n>We propose a novel bilevel optimization approach that leverages unlabeled data to interpolate between in-distribution (ID) and out-of-distribution (OOD) data.
  arXiv  Detail & Related papers  (2025-06-13T15:27:40Z)
• Semi-supervised Clustering Through Representation Learning of Large-scale EHR Data [5.591260685112265]
  SCORE is a semi-supervised representation learning framework that captures multi-domain disease profiles through patient embeddings.<n>To handle the computational challenges of large-scale data, it introduces a hybrid Expectation-Maximization (EM) and Gaussian Variational Approximation (GVA) algorithm.<n>Our analysis shows that incorporating unlabeled data enhances accuracy and reduces sensitivity to label scarcity.
  arXiv  Detail & Related papers  (2025-05-27T05:20:17Z)
• Large Language Models with Retrieval-Augmented Generation for Zero-Shot Disease Phenotyping [1.8630636381951384]
  Large language models (LLMs) offer promise in text understanding but may not efficiently handle real-world clinical documentation. We propose a zero-shot LLM-based method enriched by retrieval-augmented generation and MapReduce. We show that this method as applied to pulmonary hypertension (PH), a rare disease characterized by elevated arterial pressures in the lungs, significantly outperforms physician logic rules.
  arXiv  Detail & Related papers  (2023-12-11T15:45:27Z)
• UDAMA: Unsupervised Domain Adaptation through Multi-discriminator Adversarial Training with Noisy Labels Improves Cardio-fitness Prediction [16.26599832125242]
  We introduce UDAMA, a method with two key components: Unsupervised Domain Adaptation and Multidiscriminator Adversarial Training. In particular, we showcase the practical potential of UDAMA by applying it to Cardio-respiratory fitness (CRF) prediction. Our results show promising performance by alleviating distribution shifts in various label shift settings.
  arXiv  Detail & Related papers  (2023-07-31T13:31:53Z)
• Unsupervised EHR-based Phenotyping via Matrix and Tensor Decompositions [0.6875312133832078]
  We provide a comprehensive review of low-rank approximation-based approaches for computational phenotyping. Recent developments have adapted low-rank data approximation methods by incorporating different constraints and regularizations that facilitate interpretability further.
  arXiv  Detail & Related papers  (2022-09-01T09:47:27Z)
• Hierarchical Knowledge Guided Learning for Real-world Retinal Diseases Recognition [20.88407972858568]
  Some recently published datasets in ophthalmology AI consist of more than 40 kinds of retinal diseases with complex abnormalities and variable morbidity. From a modeling perspective, most deep learning models trained on these datasets may lack the ability to generalize to rare diseases. This paper presents a novel method that enables the deep neural network to learn from a long-tailed fundus database for various retinal disease recognition.
  arXiv  Detail & Related papers  (2021-11-17T05:44:39Z)
• Label scarcity in biomedicine: Data-rich latent factor discovery enhances phenotype prediction [102.23901690661916]
  Low-dimensional embedding spaces can be derived from the UK Biobank population dataset to enhance data-scarce prediction of health indicators, lifestyle and demographic characteristics. Performances gains from semisupervison approaches will probably become an important ingredient for various medical data science applications.
  arXiv  Detail & Related papers  (2021-10-12T16:25:50Z)
• Unsupervised Representation Learning Meets Pseudo-Label Supervised Self-Distillation: A New Approach to Rare Disease Classification [26.864435224276964]
  We propose a novel hybrid approach to rare disease classification, featuring two key novelties. First, we adopt the unsupervised representation learning (URL) based on self-supervising contrastive loss. Second, we integrate the URL with pseudo-label supervised classification for effective self-distillation of the knowledge about the rare diseases.
  arXiv  Detail & Related papers  (2021-10-09T12:56:09Z)
• Variational Knowledge Distillation for Disease Classification in Chest X-Rays [102.04931207504173]
  We propose itvariational knowledge distillation (VKD), which is a new probabilistic inference framework for disease classification based on X-rays. We demonstrate the effectiveness of our method on three public benchmark datasets with paired X-ray images and EHRs.
  arXiv  Detail & Related papers  (2021-03-19T14:13:56Z)
• Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification [83.6017225363714]
  deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance. For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming. In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
  arXiv  Detail & Related papers  (2021-02-26T02:29:30Z)
• Inheritance-guided Hierarchical Assignment for Clinical Automatic Diagnosis [50.15205065710629]
  Clinical diagnosis, which aims to assign diagnosis codes for a patient based on the clinical note, plays an essential role in clinical decision-making. We propose a novel framework to combine the inheritance-guided hierarchical assignment and co-occurrence graph propagation for clinical automatic diagnosis.
  arXiv  Detail & Related papers  (2021-01-27T13:16:51Z)
• UNITE: Uncertainty-based Health Risk Prediction Leveraging Multi-sourced Data [81.00385374948125]
  We present UNcertaInTy-based hEalth risk prediction (UNITE) model. UNITE provides accurate disease risk prediction and uncertainty estimation leveraging multi-sourced health data. We evaluate UNITE on real-world disease risk prediction tasks: nonalcoholic fatty liver disease (NASH) and Alzheimer's disease (AD) UNITE achieves up to 0.841 in F1 score for AD detection, up to 0.609 in PR-AUC for NASH detection, and outperforms various state-of-the-art baselines by up to $19%$ over the best baseline.
  arXiv  Detail & Related papers  (2020-10-22T02:28:11Z)
• Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype Prediction [55.94378672172967]
  We focus on few-shot disease subtype prediction problem, identifying subgroups of similar patients. We introduce meta learning techniques to develop a new model, which can extract the common experience or knowledge from interrelated clinical tasks. Our new model is built upon a carefully designed meta-learner, called Prototypical Network, that is a simple yet effective meta learning machine for few-shot image classification.
  arXiv  Detail & Related papers  (2020-09-02T02:50:30Z)
• Predicting Clinical Diagnosis from Patients Electronic Health Records Using BERT-based Neural Networks [62.9447303059342]
  We show the importance of this problem in medical community. We present a modification of Bidirectional Representations from Transformers (BERT) model for classification sequence. We use a large-scale Russian EHR dataset consisting of about 4 million unique patient visits.
  arXiv  Detail & Related papers  (2020-07-15T09:22:55Z)
• Semi-supervised Medical Image Classification with Relation-driven Self-ensembling Model [71.80319052891817]
  We present a relation-driven semi-supervised framework for medical image classification. It exploits the unlabeled data by encouraging the prediction consistency of given input under perturbations. Our method outperforms many state-of-the-art semi-supervised learning methods on both single-label and multi-label image classification scenarios.
  arXiv  Detail & Related papers  (2020-05-15T06:57:54Z)
• Hemogram Data as a Tool for Decision-making in COVID-19 Management: Applications to Resource Scarcity Scenarios [62.997667081978825]
  COVID-19 pandemics has challenged emergency response systems worldwide, with widespread reports of essential services breakdown and collapse of health care structure. This work describes a machine learning model derived from hemogram exam data performed in symptomatic patients. Proposed models can predict COVID-19 qRT-PCR results in symptomatic individuals with high accuracy, sensitivity and specificity.
  arXiv  Detail & Related papers  (2020-05-10T01:45:03Z)
• Deep Representation Learning of Electronic Health Records to Unlock Patient Stratification at Scale [0.5498849973527224]
  We present an unsupervised framework based on deep learning to process heterogeneous EHRs. We derive patient representations that can efficiently and effectively enable patient stratification at scale.
  arXiv  Detail & Related papers  (2020-03-14T00:04:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.