A Pseudo Label-wise Attention Network for Automatic ICD Coding

• URL: http://arxiv.org/abs/2106.06822v2
• Date: Wed, 16 Jun 2021 16:33:53 GMT
• Title: A Pseudo Label-wise Attention Network for Automatic ICD Coding
• Authors: Yifan Wu, Min Zeng, Ying Yu, Min Li
• Abstract summary: We propose a pseudo label-wise attention mechanism to tackle the problem of automatic International Classification of Diseases (ICD) coding. Instead of computing different attention modes for different ICD codes, the pseudo label-wise attention mechanism automatically merges similar ICD codes and computes only one attention mode for the similar ICD codes. Our model achieves superior performance on the public MIMIC-III dataset and private Xiangya dataset.
• Score: 17.076068093443684
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automatic International Classification of Diseases (ICD) coding is defined as a kind of text multi-label classification problem, which is difficult because the number of labels is very large and the distribution of labels is unbalanced. The label-wise attention mechanism is widely used in automatic ICD coding because it can assign weights to every word in full Electronic Medical Records (EMR) for different ICD codes. However, the label-wise attention mechanism is computational redundant and costly. In this paper, we propose a pseudo label-wise attention mechanism to tackle the problem. Instead of computing different attention modes for different ICD codes, the pseudo label-wise attention mechanism automatically merges similar ICD codes and computes only one attention mode for the similar ICD codes, which greatly compresses the number of attention modes and improves the predicted accuracy. In addition, we apply a more convenient and effective way to obtain the ICD vectors, and thus our model can predict new ICD codes by calculating the similarities between EMR vectors and ICD vectors. Extensive experiments show the superior performance of our model. On the public MIMIC-III dataset and private Xiangya dataset, our model achieves micro f1 of 0.583 and 0.806, respectively, which outperforms other competing models. Furthermore, we verify the ability of our model in predicting new ICD codes. The case study shows how pseudo label-wise attention works, and demonstrates the effectiveness of pseudo label-wise attention mechanism.

Related papers

• CoRelation: Boosting Automatic ICD Coding Through Contextualized Code Relation Learning [56.782963838838036]
  We propose a novel approach, a contextualized and flexible framework, to enhance the learning of ICD code representations. Our approach employs a dependent learning paradigm that considers the context of clinical notes in modeling all possible code relations.
  arXiv  Detail & Related papers  (2024-02-24T03:25:28Z)
• Accurate and Well-Calibrated ICD Code Assignment Through Attention Over Diverse Label Embeddings [1.201425717264024]
  Manual assigning ICD codes to clinical text is time-consuming, error-prone, and expensive. This paper describes a novel approach for automated ICD coding, combining several ideas from previous related work. Experiments with different splits of the MIMIC-III dataset show that the proposed approach outperforms the current state-of-the-art models in ICD coding.
  arXiv  Detail & Related papers  (2024-02-05T16:40:23Z)
• An Automatic ICD Coding Network Using Partition-Based Label Attention [2.371982686172067]
  We propose a novel neural network architecture composed of two parts of encoders and two kinds of label attention layers. The input text is segmentally encoded in the former encoder and integrated by the follower. Our results show that our network improves the ICD coding performance based on the partition-based mechanism.
  arXiv  Detail & Related papers  (2022-11-15T07:11:01Z)
• Revisiting Consistency Regularization for Semi-supervised Change Detection in Remote Sensing Images [60.89777029184023]
  We propose a semi-supervised CD model in which we formulate an unsupervised CD loss in addition to the supervised Cross-Entropy (CE) loss. Experiments conducted on two publicly available CD datasets show that the proposed semi-supervised CD method can reach closer to the performance of supervised CD.
  arXiv  Detail & Related papers  (2022-04-18T17:59:01Z)
• Few-Shot Electronic Health Record Coding through Graph Contrastive Learning [64.8138823920883]
  We seek to improve the performance for both frequent and rare ICD codes by using a contrastive graph-based EHR coding framework, CoGraph. CoGraph learns similarities and dissimilarities between HEWE graphs from different ICD codes so that information can be transferred among them. Two graph contrastive learning schemes, GSCL and GECL, exploit the HEWE graph structures so as to encode transferable features.
  arXiv  Detail & Related papers  (2021-06-29T14:53:17Z)
• TransICD: Transformer Based Code-wise Attention Model for Explainable ICD Coding [5.273190477622007]
  International Classification of Disease (ICD) coding procedure has been shown to be effective and crucial to the billing system in medical sector. Currently, ICD codes are assigned to a clinical note manually which is likely to cause many errors. In this project, we apply a transformer-based architecture to capture the interdependence among the tokens of a document and then use a code-wise attention mechanism to learn code-specific representations of the entire document.
  arXiv  Detail & Related papers  (2021-03-28T05:34:32Z)
• Group-aware Label Transfer for Domain Adaptive Person Re-identification [179.816105255584]
  Unsupervised Adaptive Domain (UDA) person re-identification (ReID) aims at adapting the model trained on a labeled source-domain dataset to a target-domain dataset without any further annotations. Most successful UDA-ReID approaches combine clustering-based pseudo-label prediction with representation learning and perform the two steps in an alternating fashion. We propose a Group-aware Label Transfer (GLT) algorithm, which enables the online interaction and mutual promotion of pseudo-label prediction and representation learning.
  arXiv  Detail & Related papers  (2021-03-23T07:57:39Z)
• From Extreme Multi-label to Multi-class: A Hierarchical Approach for Automated ICD-10 Coding Using Phrase-level Attention [4.387302129801651]
  Clinical coding is the task of assigning a set of alphanumeric codes, referred to as ICD (International Classification of Diseases), to a medical event based on the context captured in a clinical narrative. We propose a novel approach for automatic ICD coding by reformulating the extreme multi-label problem into a simpler multi-class problem using a hierarchical solution.
  arXiv  Detail & Related papers  (2021-02-18T03:19:14Z)
• A Label Attention Model for ICD Coding from Clinical Text [14.910833190248319]
  We propose a new label attention model for automatic ICD coding. It can handle both the various lengths and the interdependence of the ICD code related text fragments. Our model achieves new state-of-the-art results on three benchmark MIMIC datasets.
  arXiv  Detail & Related papers  (2020-07-13T12:42:43Z)
• Overcoming Statistical Shortcuts for Open-ended Visual Counting [54.858754825838865]
  We aim to develop models that learn a proper mechanism of counting regardless of the output label. First, we propose the Modifying Count Distribution protocol, which penalizes models that over-rely on statistical shortcuts. Secondly, we introduce the Spatial Counting Network (SCN), which is dedicated to visual analysis and counting based on natural language questions.
  arXiv  Detail & Related papers  (2020-06-17T18:02:01Z)
• Unsupervised Person Re-identification via Multi-label Classification [55.65870468861157]
  This paper formulates unsupervised person ReID as a multi-label classification task to progressively seek true labels. Our method starts by assigning each person image with a single-class label, then evolves to multi-label classification by leveraging the updated ReID model for label prediction. To boost the ReID model training efficiency in multi-label classification, we propose the memory-based multi-label classification loss (MMCL)
  arXiv  Detail & Related papers  (2020-04-20T12:13:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.