Probability-Biased Attention over Directed Bipartite Graphs for Long-Tail ICD Coding

• URL: http://arxiv.org/abs/2511.09559v1
• Date: Fri, 31 Oct 2025 04:47:09 GMT
• Title: Probability-Biased Attention over Directed Bipartite Graphs for Long-Tail ICD Coding
• Authors: Tianlei Chen, Yuxiao Chen, Yang Li, Feifei Wang,
• Abstract summary: We propose a learning method that models fine-grained co-occurrence relationships among codes.<n>Our method achieves state-of-the-art performance with particularly notable improvements in Macro-F1.
• Score: 12.66839524860715
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Automated International Classification of Diseases (ICD) coding aims to assign multiple disease codes to clinical documents, constituting a crucial multi-label text classification task in healthcare informatics. However, the task is challenging due to its large label space (10,000 to 20,000 codes) and long-tail distribution, where a few codes dominate while many rare codes lack sufficient training data. To address this, we propose a learning method that models fine-grained co-occurrence relationships among codes. Specifically, we construct a Directed Bipartite Graph Encoder with disjoint sets of common and rare code nodes. To facilitate a one-way information flow, edges are directed exclusively from common to rare codes. The nature of these connections is defined by a probability-based bias, which is derived from the conditional probability of a common code co-occurring given the presence of a rare code. This bias is then injected into the encoder's attention module, a process we term Co-occurrence Encoding. This structure empowers the graph encoder to enrich rare code representations by aggregating latent comorbidity information reflected in the statistical co-occurrence of their common counterparts. To ensure high-quality input to the graph, we utilize a large language model (LLM) to generate comprehensive descriptions for codes, enriching initial embeddings with clinical context and comorbidity information, serving as external knowledge for the statistical co-occurrence relationships in the code system. Experiments on three automated ICD coding benchmark datasets demonstrate that our method achieves state-of-the-art performance with particularly notable improvements in Macro-F1, which is the key metric for long-tail classification.

Related papers

• Synthetic Clinical Notes for Rare ICD Codes: A Data-Centric Framework for Long-Tail Medical Coding [26.840057002860235]
  Thousands of rare and zero-shot ICD codes are severely underrepresented in datasets like MIMIC-III.<n>We generate 90,000 synthetic notes covering 7,902 ICD codes, significantly expanding the training distribution.<n>Experiments show that our approach modestly improves macro-F1 while maintaining strong micro-F1.
  arXiv  Detail & Related papers  (2025-11-18T03:52:12Z)
• TraceCoder: Towards Traceable ICD Coding via Multi-Source Knowledge Integration [12.474362087939456]
  We propose TraceCoder, a framework integrating multi-source external knowledge to enhance traceability and explainability in ICD coding.<n>TraceCoder dynamically incorporates diverse knowledge sources, including UMLS, Wikipedia, and large language models (LLMs), to enrich code representations, bridge semantic gaps, and handle rare and ambiguous codes.<n>It also introduces a hybrid attention mechanism to model interactions among labels, clinical context, and knowledge, improving long-tail code recognition and making predictions interpretable by grounding them in external evidence.
  arXiv  Detail & Related papers  (2025-10-17T03:08:07Z)
• Is Compression Really Linear with Code Intelligence? [60.123628177110206]
  textitFormat Annealing is a lightweight, transparent training methodology designed to assess the intrinsic capabilities of pre-trained models equitably.<n>Our empirical results reveal a fundamental logarithmic relationship between measured code intelligence and bits-per-character (BPC)<n>Our work provides a more nuanced understanding of compression's role in developing code intelligence and contributes a robust evaluation framework in the code domain.
  arXiv  Detail & Related papers  (2025-05-16T16:59:14Z)
• CoRelation: Boosting Automatic ICD Coding Through Contextualized Code Relation Learning [56.782963838838036]
  We propose a novel approach, a contextualized and flexible framework, to enhance the learning of ICD code representations. Our approach employs a dependent learning paradigm that considers the context of clinical notes in modeling all possible code relations.
  arXiv  Detail & Related papers  (2024-02-24T03:25:28Z)
• A Two-Stage Decoder for Efficient ICD Coding [10.634394331433322]
  We propose a two-stage decoding mechanism to predict ICD codes. At first, we predict the parent code and then predict the child code based on the previous prediction. Experiments on the public MIMIC-III data set show that our model performs well in single-model settings.
  arXiv  Detail & Related papers  (2023-05-27T17:25:13Z)
• PEOPL: Characterizing Privately Encoded Open Datasets with Public Labels [59.66777287810985]
  We introduce information-theoretic scores for privacy and utility, which quantify the average performance of an unfaithful user. We then theoretically characterize primitives in building families of encoding schemes that motivate the use of random deep neural networks.
  arXiv  Detail & Related papers  (2023-03-31T18:03:53Z)
• HieNet: Bidirectional Hierarchy Framework for Automated ICD Coding [2.9373912230684573]
  International Classification of Diseases (ICD) is a set of classification codes for medical records. In this work, we proposed a novel Bidirectional Hierarchy Framework(HieNet) to address the challenges. Specifically, a personalized PageRank routine is developed to capture the co-relation of codes, a bidirectional hierarchy passage encoder to capture the codes' hierarchical representations, and a progressive predicting method is then proposed to narrow down the semantic searching space of prediction.
  arXiv  Detail & Related papers  (2022-12-09T14:51:12Z)
• Few-Shot Electronic Health Record Coding through Graph Contrastive Learning [64.8138823920883]
  We seek to improve the performance for both frequent and rare ICD codes by using a contrastive graph-based EHR coding framework, CoGraph. CoGraph learns similarities and dissimilarities between HEWE graphs from different ICD codes so that information can be transferred among them. Two graph contrastive learning schemes, GSCL and GECL, exploit the HEWE graph structures so as to encode transferable features.
  arXiv  Detail & Related papers  (2021-06-29T14:53:17Z)
• TransICD: Transformer Based Code-wise Attention Model for Explainable ICD Coding [5.273190477622007]
  International Classification of Disease (ICD) coding procedure has been shown to be effective and crucial to the billing system in medical sector. Currently, ICD codes are assigned to a clinical note manually which is likely to cause many errors. In this project, we apply a transformer-based architecture to capture the interdependence among the tokens of a document and then use a code-wise attention mechanism to learn code-specific representations of the entire document.
  arXiv  Detail & Related papers  (2021-03-28T05:34:32Z)
• A Meta-embedding-based Ensemble Approach for ICD Coding Prediction [64.42386426730695]
  International Classification of Diseases (ICD) are the de facto codes used globally for clinical coding. These codes enable healthcare providers to claim reimbursement and facilitate efficient storage and retrieval of diagnostic information. Our proposed approach enhances the performance of neural models by effectively training word vectors using routine medical data as well as external knowledge from scientific articles.
  arXiv  Detail & Related papers  (2021-02-26T17:49:58Z)
• A Label Attention Model for ICD Coding from Clinical Text [14.910833190248319]
  We propose a new label attention model for automatic ICD coding. It can handle both the various lengths and the interdependence of the ICD code related text fragments. Our model achieves new state-of-the-art results on three benchmark MIMIC datasets.
  arXiv  Detail & Related papers  (2020-07-13T12:42:43Z)
• Auto-Encoding Twin-Bottleneck Hashing [141.5378966676885]
  This paper proposes an efficient and adaptive code-driven graph. It is updated by decoding in the context of an auto-encoder. Experiments on benchmarked datasets clearly show the superiority of our framework over the state-of-the-art hashing methods.
  arXiv  Detail & Related papers  (2020-02-27T05:58:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.