Hierarchical Modeling of ICD Codes in EHR Foundation Models

• URL: http://arxiv.org/abs/2606.15447v1
• Date: Sat, 13 Jun 2026 19:40:27 GMT
• Title: Hierarchical Modeling of ICD Codes in EHR Foundation Models
• Abstract summary: We study ICD-10-CM hierarchy as a general inductive bias for clinical representation learning.<n>We show that explicitly encoding ICD hierarchy improves over flat code representations in both in-domain and cross-dataset settings.
• Score: 0.8620335948752808
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Electronic health record foundation models typically treat ICD diagnosis codes as flat tokens, overlooking the clinically meaningful hierarchical structure that captures disease families, subcategories, and fine-grained diagnostic detail. As a result, existing EHR representation learning methods do not explicitly exploit the hierarchical structure already present in the coding system. In this work, we study ICD-10-CM hierarchy as a general inductive bias for clinical representation learning. We investigate two complementary mechanisms for incorporating hierarchy: first, by augmenting diagnosis sequences in a BERT-style transformer with tokens corresponding to different levels of the ICD hierarchy, and second, by injecting hierarchy into graph-based code representations through hierarchy-aware edges combined with diagnosis co-occurrence structure. Across these settings, we evaluate whether explicit hierarchy improves downstream prediction, which levels of the hierarchy are most useful, whether hierarchy encoding improves transfer across datasets, and how hierarchy reshapes embedding similarity structure. We conduct experiments on two large-scale real-world clinical datasets: MIMIC-IV, used for pretraining and in-domain evaluation, and eICU, used to assess cross-dataset transfer via frozen encoder probing. Our findings show that explicitly encoding ICD hierarchy improves over flat code representations in both in-domain and cross-dataset settings, while revealing that the most useful level of hierarchy depends on both the task and the modeling approach. More broadly, we focus on hierarchy-aware EHR representation learning and show that the benefits of encoding hierarchy are generalizable across modeling settings and hierarchy levels.

Related papers

• Online Continual Learning with Dynamic Label Hierarchies [72.33335166136045]
  We introduce a new problem setting, DHOCL, where severity evolve across granularities and each sample provides supervision at a single hierarchical level.<n>In this setting, we find two fundamental issues: (i) partial supervision under mixed granularities provides only point-wise signals over an evolving path-wise hierarchy, which constrains plasticity and undermines cross-level semantic consistency.<n>We propose HALO, which adaptively combines complementary classification heads, regularized by organized learnable hierarchical prototypes, enabling rapid adaptation, hierarchical consistency, and structured knowledge consolidation as the taxonomy evolves.
  arXiv  Detail & Related papers  (2026-05-12T08:20:23Z)
• A Semantics-Aware Hierarchical Self-Supervised Approach to Classification of Remote Sensing Images [12.282079123411947]
  We present a novel Semantics-Aware Hierarchical Consensus (SAHC) method for learning hierarchical features and relationships.<n>The SAHC method is evaluated on three benchmark datasets with different degrees of hierarchical complexity.<n> Experimental results show both the effectiveness of the proposed approach in guiding network learning and the robustness of the hierarchical consensus for remote sensing image classification tasks.
  arXiv  Detail & Related papers  (2025-10-06T15:30:39Z)
• Feature Identification for Hierarchical Contrastive Learning [7.655211354400059]
  We propose two novel hierarchical contrastive learning (HMLC) methods.<n>Our approach explicitly models inter-class relationships and imbalanced class distribution at higher hierarchy levels.<n>Our method achieves state-of-the-art performance in linear evaluation, outperforming existing hierarchical contrastive learning methods by 2 percentage points in terms of accuracy.
  arXiv  Detail & Related papers  (2025-10-01T12:46:47Z)
• Deep Taxonomic Networks for Unsupervised Hierarchical Prototype Discovery [5.300910554558862]
  Existing methods often tie the structure to the number of classes and underutilize the rich prototype information available at intermediate hierarchical levels.<n>We introduce deep taxonomic networks, a novel deep latent variable approach designed to bridge these gaps.
  arXiv  Detail & Related papers  (2025-09-28T03:13:32Z)
• Hierarchical knowledge guided fault intensity diagnosis of complex industrial systems [9.830457900441939]
  We propose a hierarchical knowledge guided fault intensity diagnosis framework (HKG) inspired by the tree of thought.<n>The HKG uses graph convolutional networks to map the hierarchical topological graph of class representations into a set of global hierarchical classifiers.<n>In addition, we develop a re-weighted hierarchical knowledge correlation matrix (Re-HKCM) scheme by embedding inter-class hierarchical knowledge into a data-driven statistical correlation matrix.
  arXiv  Detail & Related papers  (2025-08-17T14:22:03Z)
• Hyperbolic Residual Quantization: Discrete Representations for Data with Latent Hierarchies [48.72319569157807]
  Residual Quantization (RQ) is widely used to generate discrete, multitoken representations for hierarchical data.<n>We propose Hyperbolic Residual Quantization (HRQ), which embeds data in a hyperbolic manifold.<n>HRQ imparts an inductive bias that aligns naturally with hierarchical branching.
  arXiv  Detail & Related papers  (2025-05-18T13:14:07Z)
• Implant Global and Local Hierarchy Information to Sequence based Code Representation Models [25.776540440893257]
  We analyze how the complete hierarchical structure influences the tokens in code sequences and abstract this influence as a property of code tokens called hierarchical embedding. We propose the Hierarchy Transformer (HiT), a simple but effective sequence model to incorporate the complete hierarchical embeddings of source code into a Transformer model.
  arXiv  Detail & Related papers  (2023-03-14T12:01:39Z)
• Deep Hierarchical Semantic Segmentation [76.40565872257709]
  hierarchical semantic segmentation (HSS) aims at structured, pixel-wise description of visual observation in terms of a class hierarchy. HSSN casts HSS as a pixel-wise multi-label classification task, only bringing minimal architecture change to current segmentation models. With hierarchy-induced margin constraints, HSSN reshapes the pixel embedding space, so as to generate well-structured pixel representations.
  arXiv  Detail & Related papers  (2022-03-27T15:47:44Z)
• CoPHE: A Count-Preserving Hierarchical Evaluation Metric in Large-Scale Multi-Label Text Classification [70.554573538777]
  We argue for hierarchical evaluation of the predictions of neural LMTC models. We describe a structural issue in the representation of the structured label space in prior art. We propose a set of metrics for hierarchical evaluation using the depth-based representation.
  arXiv  Detail & Related papers  (2021-09-10T13:09:12Z)
• HUMAP: Hierarchical Uniform Manifold Approximation and Projection [40.77787659104315]
  This work presents HUMAP, a novel hierarchical dimensionality reduction technique designed to be flexible on preserving local and global structures.<n>We provide empirical evidence of our technique's superiority compared with current hierarchical approaches and show a case study applying HUMAP for dataset labelling.
  arXiv  Detail & Related papers  (2021-06-14T19:27:54Z)
• A Meta-embedding-based Ensemble Approach for ICD Coding Prediction [64.42386426730695]
  International Classification of Diseases (ICD) are the de facto codes used globally for clinical coding. These codes enable healthcare providers to claim reimbursement and facilitate efficient storage and retrieval of diagnostic information. Our proposed approach enhances the performance of neural models by effectively training word vectors using routine medical data as well as external knowledge from scientific articles.
  arXiv  Detail & Related papers  (2021-02-26T17:49:58Z)
• Dual-constrained Deep Semi-Supervised Coupled Factorization Network with Enriched Prior [80.5637175255349]
  We propose a new enriched prior based Dual-constrained Deep Semi-Supervised Coupled Factorization Network, called DS2CF-Net. To ex-tract hidden deep features, DS2CF-Net is modeled as a deep-structure and geometrical structure-constrained neural network. Our network can obtain state-of-the-art performance for representation learning and clustering.
  arXiv  Detail & Related papers  (2020-09-08T13:10:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.