Knowledge Injected Prompt Based Fine-tuning for Multi-label Few-shot ICD Coding

• URL: http://arxiv.org/abs/2210.03304v1
• Date: Fri, 7 Oct 2022 03:25:58 GMT
• Title: Knowledge Injected Prompt Based Fine-tuning for Multi-label Few-shot ICD Coding
• Abstract summary: This study addresses the long-tail challenge by adapting a prompt-based fine-tuning technique with label semantics. Experiments on MIMIC-III-full, a benchmark dataset of code assignment, show that our proposed method outperforms previous state-of-art method in 14.5% in marco F1. Our model improves marco F1 from 17.1 to 30.4 and micro F1 from 17.2 to 32.6 compared to previous method.
• Score: 7.8183215844641
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Automatic International Classification of Diseases (ICD) coding aims to assign multiple ICD codes to a medical note with average length of 3,000+ tokens. This task is challenging due to a high-dimensional space of multi-label assignment (tens of thousands of ICD codes) and the long-tail challenge: only a few codes (common diseases) are frequently assigned while most codes (rare diseases) are infrequently assigned. This study addresses the long-tail challenge by adapting a prompt-based fine-tuning technique with label semantics, which has been shown to be effective under few-shot setting. To further enhance the performance in medical domain, we propose a knowledge-enhanced longformer by injecting three domain-specific knowledge: hierarchy, synonym, and abbreviation with additional pretraining using contrastive learning. Experiments on MIMIC-III-full, a benchmark dataset of code assignment, show that our proposed method outperforms previous state-of-the-art method in 14.5% in marco F1 (from 10.3 to 11.8, P<0.001). To further test our model on few-shot setting, we created a new rare diseases coding dataset, MIMIC-III-rare50, on which our model improves marco F1 from 17.1 to 30.4 and micro F1 from 17.2 to 32.6 compared to previous method.

Related papers

• Understanding the Limits of Agentic ICD Coding [18.80812324281645]
  ICD-10-CM codes are alphanumeric codes used in the US to classify diagnoses and injuries for medical billing and epidemiological reporting.<n>Standard ICD-10-CM benchmarks report aggregate metrics that obscure performance on complex coding scenarios.<n>We evaluate neural, workflow, and agentic systems on a rarity-stratified set of MIMIC-IV discharge summaries.
  arXiv  Detail & Related papers  (2026-09-12T08:33:38Z)
• Deep Label-Wise Attentive Temporal Convolutional Networks Improve Medical Coding [54.92174500158608]
  We model the task as a multi-label text classification problem.<n>We propose a deep neural model consisting of a multi-layer temporal convolution network (TCN) followed by label-wise attention.<n>Our method achieves significantly better F-1 scores (9% increase) compared to the previous state-of-the-art model, with a remarkable increase in recall score (28% increase)
  arXiv  Detail & Related papers  (2026-07-27T22:48:05Z)
• Synthetic Clinical Notes for Rare ICD Codes: A Data-Centric Framework for Long-Tail Medical Coding [26.840057002860235]
  Thousands of rare and zero-shot ICD codes are severely underrepresented in datasets like MIMIC-III.<n>We generate 90,000 synthetic notes covering 7,902 ICD codes, significantly expanding the training distribution.<n>Experiments show that our approach modestly improves macro-F1 while maintaining strong micro-F1.
  arXiv  Detail & Related papers  (2025-11-18T03:52:12Z)
• ACE-ICD: Acronym Expansion As Data Augmentation For Automated ICD Coding [2.0267433264426598]
  We propose a novel effective data augmentation technique that leverages large language models to expand medical acronyms.<n>Our approach, ACE-ICD establishes new state-of-the-art performance across multiple settings, including common codes, rare codes, and full-code assignments.
  arXiv  Detail & Related papers  (2025-11-10T17:11:20Z)
• Probability-Biased Attention over Directed Bipartite Graphs for Long-Tail ICD Coding [12.66839524860715]
  We propose a learning method that models fine-grained co-occurrence relationships among codes.<n>Our method achieves state-of-the-art performance with particularly notable improvements in Macro-F1.
  arXiv  Detail & Related papers  (2025-10-31T04:47:09Z)
• MedSeqFT: Sequential Fine-tuning Foundation Models for 3D Medical Image Segmentation [55.37355146924576]
  MedSeqFT is a sequential fine-tuning framework for medical image analysis.<n>It adapts pre-trained models to new tasks while refining their representational capacity.<n>It consistently outperforms state-of-the-art fine-tuning strategies.
  arXiv  Detail & Related papers  (2025-09-07T15:22:53Z)
• MedGemma Technical Report [75.88152277443179]
  We introduce MedGemma, a collection of medical vision-language foundation models based on Gemma 3 4B and 27B.<n>MedGemma demonstrates advanced medical understanding and reasoning on images and text.<n>We additionally introduce MedSigLIP, a medically-tuned vision encoder derived from SigLIP.
  arXiv  Detail & Related papers  (2025-07-07T17:01:44Z)
• Evaluating Hierarchical Clinical Document Classification Using Reasoning-Based LLMs [7.026393789313748]
  This study evaluates how well large language models (LLMs) can classify ICD-10 codes from hospital discharge summaries.<n> Reasoning-based models generally outperformed non-reasoning ones, with Gemini 2.5 Pro performing best overall.
  arXiv  Detail & Related papers  (2025-07-02T00:53:54Z)
• PMT: Progressive Mean Teacher via Exploring Temporal Consistency for Semi-Supervised Medical Image Segmentation [51.509573838103854]
  We propose a semi-supervised learning framework, termed Progressive Mean Teachers (PMT), for medical image segmentation. Our PMT generates high-fidelity pseudo labels by learning robust and diverse features in the training process. Experimental results on two datasets with different modalities, i.e., CT and MRI, demonstrate that our method outperforms the state-of-the-art medical image segmentation approaches.
  arXiv  Detail & Related papers  (2024-09-08T15:02:25Z)
• Auxiliary Knowledge-Induced Learning for Automatic Multi-Label Medical Document Classification [22.323705343864336]
  We propose a novel approach for ICD indexing that adopts three ideas. We use a multi-level deep dilated residual convolution encoder to aggregate the information from the clinical notes. We formalize the task of ICD classification with auxiliary knowledge of the medical records.
  arXiv  Detail & Related papers  (2024-05-29T13:44:07Z)
• LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching [59.01894976615714]
  We introduce LVM-Med, the first family of deep networks trained on large-scale medical datasets. We have collected approximately 1.3 million medical images from 55 publicly available datasets. LVM-Med empirically outperforms a number of state-of-the-art supervised, self-supervised, and foundation models.
  arXiv  Detail & Related papers  (2023-06-20T22:21:34Z)
• Automated Medical Coding on MIMIC-III and MIMIC-IV: A Critical Review and Replicability Study [60.56194508762205]
  We reproduce, compare, and analyze state-of-the-art automated medical coding machine learning models. We show that several models underperform due to weak configurations, poorly sampled train-test splits, and insufficient evaluation. We present the first comprehensive results on the newly released MIMIC-IV dataset using the reproduced models.
  arXiv  Detail & Related papers  (2023-04-21T11:54:44Z)
• Multi-label Few-shot ICD Coding as Autoregressive Generation with Prompt [7.554528566861559]
  This study transforms this multi-label classification task into an autoregressive generation task. Instead of directly predicting the high dimensional space of ICD codes, our model generates the lower dimension of text descriptions. Experiments on MIMIC-III-few show that our model performs with a marco F1 30.2, which substantially outperforms the previous MIMIC-III-full SOTA model.
  arXiv  Detail & Related papers  (2022-11-24T22:10:50Z)
• ICDBigBird: A Contextual Embedding Model for ICD Code Classification [71.58299917476195]
  Contextual word embedding models have achieved state-of-the-art results in multiple NLP tasks. ICDBigBird is a BigBird-based model which can integrate a Graph Convolutional Network (GCN) Our experiments on a real-world clinical dataset demonstrate the effectiveness of our BigBird-based model on the ICD classification task.
  arXiv  Detail & Related papers  (2022-04-21T20:59:56Z)
• Statistical Dependency Guided Contrastive Learning for Multiple Labeling in Prenatal Ultrasound [56.631021151764955]
  Standard plane recognition plays an important role in prenatal ultrasound (US) screening. We build a novel multi-label learning scheme to identify multiple standard planes and corresponding anatomical structures simultaneously.
  arXiv  Detail & Related papers  (2021-08-11T06:39:26Z)
• Medical Code Prediction from Discharge Summary: Document to Sequence BERT using Sequence Attention [0.0]
  We propose a model based on bidirectional encoder representations from transformer (BERT) using the sequence attention method for automatic ICD code assignment. We evaluate our ap-proach on the MIMIC-III benchmark dataset.
  arXiv  Detail & Related papers  (2021-06-15T07:35:50Z)
• From Extreme Multi-label to Multi-class: A Hierarchical Approach for Automated ICD-10 Coding Using Phrase-level Attention [4.387302129801651]
  Clinical coding is the task of assigning a set of alphanumeric codes, referred to as ICD (International Classification of Diseases), to a medical event based on the context captured in a clinical narrative. We propose a novel approach for automatic ICD coding by reformulating the extreme multi-label problem into a simpler multi-class problem using a hierarchical solution.
  arXiv  Detail & Related papers  (2021-02-18T03:19:14Z)
• Inheritance-guided Hierarchical Assignment for Clinical Automatic Diagnosis [50.15205065710629]
  Clinical diagnosis, which aims to assign diagnosis codes for a patient based on the clinical note, plays an essential role in clinical decision-making. We propose a novel framework to combine the inheritance-guided hierarchical assignment and co-occurrence graph propagation for clinical automatic diagnosis.
  arXiv  Detail & Related papers  (2021-01-27T13:16:51Z)
• A Label Attention Model for ICD Coding from Clinical Text [14.910833190248319]
  We propose a new label attention model for automatic ICD coding. It can handle both the various lengths and the interdependence of the ICD code related text fragments. Our model achieves new state-of-the-art results on three benchmark MIMIC datasets.
  arXiv  Detail & Related papers  (2020-07-13T12:42:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.