From Documents to Spans: Code-Centric Learning for LLM-based ICD Coding

• URL: http://arxiv.org/abs/2603.15270v1
• Date: Mon, 16 Mar 2026 13:37:01 GMT
• Title: From Documents to Spans: Code-Centric Learning for LLM-based ICD Coding
• Abstract summary: We propose Code-Centric Learning, a training framework that shifts supervision from full clinical documents to scalable, short evidence spans.<n>Our proposed framework consists of a mixed training strategy and code-centric data expansion, which substantially reduces training cost, improves accuracy on unseen ICD codes and preserves interpretability.
• Score: 29.729356191729888
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: ICD coding is a critical yet challenging task in healthcare. Recently, LLM-based methods demonstrate stronger generalization than discriminative methods in ICD coding. However, fine-tuning LLMs for ICD coding faces three major challenges. First, existing public ICD coding datasets provide limited coverage of the ICD code space, restricting a model's ability to generalize to unseen codes. Second, naive fine-tuning diminishes the interpretability of LLMs, as few public datasets contain explicit supporting evidence for assigned codes. Third, ICD coding typically involves long clinical documents, making fine-tuning LLMs computationally expensive. To address these issues, we propose Code-Centric Learning, a training framework that shifts supervision from full clinical documents to scalable, short evidence spans. The key idea of this framework is that span-level learning improves LLMs' ability to perform document-level ICD coding. Our proposed framework consists of a mixed training strategy and code-centric data expansion, which substantially reduces training cost, improves accuracy on unseen ICD codes and preserves interpretability. Under the same LLM backbone, our method substantially outperforms strong baselines. Notably, our method enables small-scale LLMs to achieve performance comparable to much larger proprietary models, demonstrating its effectiveness and potential for fully automated ICD coding.

Related papers

• KREL: Automatic Medical Coding via Knowledge-Guided Reasoning over Clinical Evidence with LLMs [32.666218878368575]
  Automatic Medical Coding (AMC) assigns standardized International Classification of Diseases (ICD) codes to clinical notes.<n>Key challenges remain, including the extreme length of clinical notes and the vast ICD label space.<n>We propose Knowledge-Guided Reasoning over Clinical Evidence with LLMs.
  arXiv  Detail & Related papers  (2026-08-21T09:05:15Z)
• Can Post-Training Turn LLMs into Good Medical Coders? An Empirical Study of Generative ICD Coding [40.63304781183767]
  Generative large language models (LLMs) are often reported as weak medical coders.<n>We present a controlled empirical study of post-training for generative ICD coding.
  arXiv  Detail & Related papers  (2026-06-11T22:04:50Z)
• CodeSimpleQA: Scaling Factuality in Code Large Language Models [55.705748501461294]
  We present CodeSimpleQA, a comprehensive benchmark designed to evaluate the factual accuracy of code LLMs in answering code-related questions.<n>We also create CodeSimpleQA-Instruct, a large-scale instruction corpus with 66M samples, and develop a post-training framework combining supervised fine-tuning and reinforcement learning.
  arXiv  Detail & Related papers  (2025-12-22T14:27:17Z)
• From Code Foundation Models to Agents and Applications: A Practical Guide to Code Intelligence [150.3696990310269]
  Large language models (LLMs) have transformed automated software development by enabling direct translation of natural language descriptions into functional code.<n>We provide a comprehensive synthesis and practical guide (a series of analytic and probing experiments) about code LLMs.<n>We analyze the code capability of the general LLMs (GPT-4, Claude, LLaMA) and code-specialized LLMs (StarCoder, Code LLaMA, DeepSeek-Coder, and QwenCoder)
  arXiv  Detail & Related papers  (2025-11-23T17:09:34Z)
• Is Compression Really Linear with Code Intelligence? [60.123628177110206]
  textitFormat Annealing is a lightweight, transparent training methodology designed to assess the intrinsic capabilities of pre-trained models equitably.<n>Our empirical results reveal a fundamental logarithmic relationship between measured code intelligence and bits-per-character (BPC)<n>Our work provides a more nuanced understanding of compression's role in developing code intelligence and contributes a robust evaluation framework in the code domain.
  arXiv  Detail & Related papers  (2025-05-16T16:59:14Z)
• Post-Incorporating Code Structural Knowledge into Pretrained Models via ICL for Code Translation [15.73070332657629]
  Large language models (LLMs) have achieved significant advancements in software mining.<n> handling the syntactic structure of source code remains a challenge.<n>This paper employs incontext learning (ICL) to integrate code structural knowledge into pre-trained LLMs.
  arXiv  Detail & Related papers  (2025-03-28T10:59:42Z)
• OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models [76.59316249991657]
  Large language models (LLMs) for code have become indispensable in various domains, including code generation, reasoning tasks and agent systems.<n>While open-access code LLMs are increasingly approaching the performance levels of proprietary models, high-quality code LLMs remain limited.<n>We introduce OpenCoder, a top-tier code LLM that not only achieves performance comparable to leading models but also serves as an "open cookbook" for the research community.
  arXiv  Detail & Related papers  (2024-11-07T17:47:25Z)
• Crystal: Illuminating LLM Abilities on Language and Code [58.5467653736537]
  We propose a pretraining strategy to enhance the integration of natural language and coding capabilities. The resulting model, Crystal, demonstrates remarkable capabilities in both domains.
  arXiv  Detail & Related papers  (2024-11-06T10:28:46Z)
• What's Wrong with Your Code Generated by Large Language Models? An Extensive Study [92.62952504133926]
  This study evaluated the performance of three leading closed-source LLMs and six popular open-source LLMs on three commonly used benchmarks.<n>We developed a taxonomy of bugs for incorrect codes and analyzed the root cause for common bug types.<n>We propose a novel training-free iterative method that introduces self-critique, enabling LLMs to critique and correct their generated code.
  arXiv  Detail & Related papers  (2024-07-08T17:27:17Z)
• Exploring LLM Multi-Agents for ICD Coding [15.730751450511333]
  The proposed multi-agent method for ICD coding effectively mimics the real-world coding process and improves performance on both common and rare codes. Our method achieves comparable results to state-of-the-art ICD coding methods that require extensive pre-training or fine-tuning, and outperforms them in rare code accuracy, and explainability.
  arXiv  Detail & Related papers  (2024-04-01T15:17:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.