Semantic Decomposition Improves Learning of Large Language Models on EHR Data

• URL: http://arxiv.org/abs/2212.06040v1
• Date: Mon, 14 Nov 2022 14:59:16 GMT
• Title: Semantic Decomposition Improves Learning of Large Language Models on EHR Data
• Abstract summary: H-BERT ingests complete graph tree expansions of hierarchical medical codes as opposed to only ingesting the leaves and pushes patient-level labels down to each visit. This methodology significantly improves prediction of patient membership in over 500 medical diagnosis classes as measured by aggregated AUC and APS.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Electronic health records (EHR) are widely believed to hold a profusion of actionable insights, encrypted in an irregular, semi-structured format, amidst a loud noise background. To simplify learning patterns of health and disease, medical codes in EHR can be decomposed into semantic units connected by hierarchical graphs. Building on earlier synergy between Bidirectional Encoder Representations from Transformers (BERT) and Graph Attention Networks (GAT), we present H-BERT, which ingests complete graph tree expansions of hierarchical medical codes as opposed to only ingesting the leaves and pushes patient-level labels down to each visit. This methodology significantly improves prediction of patient membership in over 500 medical diagnosis classes as measured by aggregated AUC and APS, and creates distinct representations of patients in closely related but clinically distinct phenotypes.

Related papers

• REFINE: LLM Refinement over Budgeted Text-Attributed Graphs for Personalized Medical Concept Representation [12.878616211892115]
  REFINE is a KG-aware budgeted graph refinement framework for patient-personalized medical concept encoding.<n> REFINE consistently improves diverse EHR backbones, outperforms strong baselines, and demonstrates robust gains across component ablation, KG selection, and data insufficiency.
  arXiv  Detail & Related papers  (2026-09-03T19:29:22Z)
• Multimodal Graph-based Classification of Esophageal Motility Disorders [73.90451172929117]
  Diagnosing esophageal motility disorders pose significant challenges due to the complexity of high-resolution impedance manometry data and variability in clinical interpretation.<n>This work explores the feasibility of a multimodal Machine Learning-based classification approach that combines HRIM recordings with patient-specific information and incorporates a graph-based modeling of esophageal physiology.
  arXiv  Detail & Related papers  (2026-05-13T14:52:12Z)
• Text-Attributed Knowledge Graph Enrichment with Large Language Models for Medical Concept Representation [14.711294299248024]
  We present CoMed, an LLM-empowered graph learning framework for medical concept representation.<n>CoMed first builds a global knowledge graph (KG) over medical codes by combining statistically reliable associations mined from EHRs with type-constrained LLM.<n>It then utilizes LLMs to enrich the KG into a text-attributed graph by generating node descriptions and edge rationales.<n>Finally, CoMed jointly trains a LoRA-tuned LLaMA text encoder with a heterogeneous GNN, fusing text semantics and graph structure into unified concept embeddings.
  arXiv  Detail & Related papers  (2026-04-14T22:29:18Z)
• DT-BEHRT: Disease Trajectory-aware Transformer for Interpretable Patient Representation Learning [29.777694695551606]
  Disease Trajectory-aware Transformer for EHR (DT-BEHRT)<n>We propose a graph-enhanced sequential architecture that disentangles disease trajectories by explicitly modeling diagnosis-centric interactions within organ systems and capturing asynchronous progression patterns.<n>Experiments on multiple benchmark datasets demonstrate that DT-BEHRT achieves strong predictive performance and provides interpretable patient representations that align with clinicians' disease-centered reasoning.
  arXiv  Detail & Related papers  (2026-03-10T19:21:40Z)
• Sepsis Prediction Using Graph Convolutional Networks over Patient-Feature-Value Triplets [10.151360630975482]
  We propose Triplet-GCN, a single-branch graph convolutional model that represents each encounter as patient--feature--value triplets.<n>We learn patient embeddings via a Graph Convolutional Network (GCN) followed by a lightweight multilayer perceptron (MLP)
  arXiv  Detail & Related papers  (2025-12-05T04:30:48Z)
• MIRNet: Integrating Constrained Graph-Based Reasoning with Pre-training for Diagnostic Medical Imaging [67.74482877175797]
  MIRNet is a novel framework that integrates self-supervised pre-training with constrained graph-based reasoning.<n>We introduce TongueAtlas-4K, a benchmark comprising 4,000 images annotated with 22 diagnostic labels.
  arXiv  Detail & Related papers  (2025-11-13T06:30:41Z)
• Automated Hierarchical Graph Construction for Multi-source Electronic Health Records [17.122817545326928]
  We propose MASH, a fully automated framework that aligns medical codes across institutions using neural optimal transport.<n>MASH integrates information from pre-trained language models, co-occurrence patterns, textual descriptions, and supervised labels.<n>It produces interpretable hierarchical graphs that facilitate the navigation and understanding of heterogeneous clinical data.
  arXiv  Detail & Related papers  (2025-09-08T11:45:59Z)
• Generating Clinically Realistic EHR Data via a Hierarchy- and Semantics-Guided Transformer [0.0]
  We propose the Hierarchy- and Semantics-Guided Transformer (HiSGT), a novel framework for the generative process. HiSGT constructs a hierarchical graph to encode parent-child and sibling relationships among clinical codes and employs a graph neural network to derive hierarchy-aware embeddings. Experiments on the MIMIC-III and MIMIC-IV datasets demonstrate that HiSGT significantly improves the statistical alignment of synthetic data with real patient records.
  arXiv  Detail & Related papers  (2025-02-28T05:06:04Z)
• DualMAR: Medical-Augmented Representation from Dual-Expertise Perspectives [20.369746122143063]
  We propose DualMAR, a framework that enhances prediction tasks through both individual observation data and public knowledge bases. By retrieving and angular coordinates upon polar space, DualMAR enables accurate predictions based on rich hierarchical and semantic embeddings from KG.
  arXiv  Detail & Related papers  (2024-10-25T20:25:22Z)
• Predictive Modeling with Temporal Graphical Representation on Electronic Health Records [8.996666837088311]
  An effective representation of a patient's EHR should encompass both the temporal relationships between historical visits and medical events. We model a patient's EHR as a novel temporal heterogeneous graph. It propagates structured information from medical event nodes to visit nodes and utilizes time-aware visit nodes to capture changes in the patient's health status.
  arXiv  Detail & Related papers  (2024-05-07T02:05:30Z)
• Multimodal Fusion of EHR in Structures and Semantics: Integrating Clinical Records and Notes with Hypergraph and LLM [39.25272553560425]
  We propose a new framework called MINGLE, which integrates both structures and semantics in EHR effectively. Our framework uses a two-level infusion strategy to combine medical concept semantics and clinical note semantics into hypergraph neural networks. Experiment results on two EHR datasets, the public MIMIC-III and private CRADLE, show that MINGLE can effectively improve predictive performance by 11.83% relatively.
  arXiv  Detail & Related papers  (2024-02-19T23:48:40Z)
• Next Visit Diagnosis Prediction via Medical Code-Centric Multimodal Contrastive EHR Modelling with Hierarchical Regularisation [0.0]
  We propose NECHO, a novel medical code-centric multimodal contrastive EHR learning framework with hierarchical regularisation. First, we integrate multifaceted information encompassing medical codes, demographics, and clinical notes using a tailored network design. We also regularise modality-specific encoders using a parental level information in medical ontology to learn hierarchical structure of EHR data.
  arXiv  Detail & Related papers  (2024-01-22T01:58:32Z)
• Customizing General-Purpose Foundation Models for Medical Report Generation [64.31265734687182]
  The scarcity of labelled medical image-report pairs presents great challenges in the development of deep and large-scale neural networks. We propose customizing off-the-shelf general-purpose large-scale pre-trained models, i.e., foundation models (FMs) in computer vision and natural language processing.
  arXiv  Detail & Related papers  (2023-06-09T03:02:36Z)
• Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generation [92.73584302508907]
  We propose a knowledge graph with Dynamic structure and nodes to facilitate medical report generation with Contrastive Learning. In detail, the fundamental structure of our graph is pre-constructed from general knowledge. Each image feature is integrated with its very own updated graph before being fed into the decoder module for report generation.
  arXiv  Detail & Related papers  (2023-03-18T03:53:43Z)
• MIMO: Mutual Integration of Patient Journey and Medical Ontology for Healthcare Representation Learning [49.57261599776167]
  We propose an end-to-end robust Transformer-based solution, Mutual Integration of patient journey and Medical Ontology (MIMO) for healthcare representation learning and predictive analytics.
  arXiv  Detail & Related papers  (2021-07-20T07:04:52Z)
• G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for Biomarker Identification and Disease Classification [49.53651166356737]
  We propose a novel deep neural network architecture to integrate imaging and genetics data, as guided by diagnosis, that provides interpretable biomarkers. We have evaluated our model on a population study of schizophrenia that includes two functional MRI (fMRI) paradigms and Single Nucleotide Polymorphism (SNP) data.
  arXiv  Detail & Related papers  (2021-01-27T19:28:04Z)
• BiteNet: Bidirectional Temporal Encoder Network to Predict Medical Outcomes [53.163089893876645]
  We propose a novel self-attention mechanism that captures the contextual dependency and temporal relationships within a patient's healthcare journey. An end-to-end bidirectional temporal encoder network (BiteNet) then learns representations of the patient's journeys. We have evaluated the effectiveness of our methods on two supervised prediction and two unsupervised clustering tasks with a real-world EHR dataset.
  arXiv  Detail & Related papers  (2020-09-24T00:42:36Z)
• Deep Representation Learning of Electronic Health Records to Unlock Patient Stratification at Scale [0.5498849973527224]
  We present an unsupervised framework based on deep learning to process heterogeneous EHRs. We derive patient representations that can efficiently and effectively enable patient stratification at scale.
  arXiv  Detail & Related papers  (2020-03-14T00:04:20Z)
• Learning Dynamic and Personalized Comorbidity Networks from Event Data using Deep Diffusion Processes [102.02672176520382]
  Comorbid diseases co-occur and progress via complex temporal patterns that vary among individuals. In electronic health records we can observe the different diseases a patient has, but can only infer the temporal relationship between each co-morbid condition. We develop deep diffusion processes to model "dynamic comorbidity networks"
  arXiv  Detail & Related papers  (2020-01-08T15:47:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.