Training a Large Language Model for Medical Coding Using Privacy-Preserving Synthetic Clinical Data

• URL: http://arxiv.org/abs/2603.23515v1
• Date: Fri, 06 Mar 2026 02:02:49 GMT
• Title: Training a Large Language Model for Medical Coding Using Privacy-Preserving Synthetic Clinical Data
• Abstract summary: We investigate whether a modern open-weight foundation model can be adapted for an expert-level medical coding task.<n>We fine-tune Llama 3-70B on pairs of clinical notes and gold codes generated from EHR-grounded templates and coding policies.<n>After fine-tuning on the synthetic corpus, exact-match F1 exceeded 0.70, representing a large absolute gain across both code systems.
• Score: 10.24013423253024
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Improving the accuracy and reliability of medical coding reduces clinician burnout and supports revenue cycle processes, freeing providers to focus more on patient care. However, automating the assignment of ICD-10-CM and CPT codes from clinical documentation remains a challenge due to heterogeneous records, nuanced coding guidelines, and long-tail distributions. Large language models have been proposed to help or automate specific medical coding tasks. However, foundation models are not explicitly trained for medical coding and zero-shot coding has yielded poor results. We investigate whether a modern open-weight foundation model can be adapted for an expert-level medical coding task using privacy-preserving synthetic training data derived from electronic health records. We fine-tune Llama 3-70B on pairs of clinical notes and gold codes generated from EHR-grounded templates and coding policies, then evaluate exact-code prediction for ICD-10-CM and CPT. A zero-shot baseline with the unadapted model achieved an F1 score of 0.18 for exact code match. After fine-tuning on the synthetic corpus, exact-match F1 exceeded 0.70, representing a large absolute gain across both code systems. Notably, performance remained high on complex categories that often require multi-step clinical reasoning and code composition, including Advanced Illness and Frailty classes, and the model retained its performance on medical comprehension tasks. These results indicate that synthetic, policy-aware data can efficiently teach a general-purpose large language model to support precise medical coding without exposing protected health information. The approach offers a practical path for training coding agents safely and iteratively on specific tasks that represent real-world populations.

Related papers

• Deep Label-Wise Attentive Temporal Convolutional Networks Improve Medical Coding [54.92174500158608]
  We model the task as a multi-label text classification problem.<n>We propose a deep neural model consisting of a multi-layer temporal convolution network (TCN) followed by label-wise attention.<n>Our method achieves significantly better F-1 scores (9% increase) compared to the previous state-of-the-art model, with a remarkable increase in recall score (28% increase)
  arXiv  Detail & Related papers  (2026-07-27T22:48:05Z)
• Leveraging Large Language Models to Extract and Translate Medical Information in Doctors' Notes for Health Records and Diagnostic Billing Codes [0.0]
  This thesis explores using open-weight Large Language Models (LLMs) to extract clinical information from physician notes and translate it into ICD-10-CM diagnostic codes without reliance on cloud-based services.<n>A privacy-focused pipeline was developed using Ollama, LangChain, and containerized environments to evaluate multiple open-weight models.<n>Results show that strict schema enforcement achieved near 100% compliance, but accurate generation of specific diagnostic codes remains challenging for smaller models.
  arXiv  Detail & Related papers  (2026-01-14T12:21:12Z)
• MedDCR: Learning to Design Agentic Workflows for Medical Coding [55.51674334874892]
  Medical coding converts free-text clinical notes into standardized diagnostic and procedural codes.<n>We present MedDCR, a closed-loop framework that treats design as a learning problem.<n>On benchmark datasets, MedDCR outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-11-17T13:30:51Z)
• Evaluating Hierarchical Clinical Document Classification Using Reasoning-Based LLMs [7.026393789313748]
  This study evaluates how well large language models (LLMs) can classify ICD-10 codes from hospital discharge summaries.<n> Reasoning-based models generally outperformed non-reasoning ones, with Gemini 2.5 Pro performing best overall.
  arXiv  Detail & Related papers  (2025-07-02T00:53:54Z)
• Large language models are good medical coders, if provided with tools [0.0]
  This study presents a novel two-stage Retrieve-Rank system for automated ICD-10-CM medical coding. evaluating both systems on a dataset of 100 single-term medical conditions. The Retrieve-Rank system achieved 100% accuracy in predicting correct ICD-10-CM codes.
  arXiv  Detail & Related papers  (2024-07-06T06:58:51Z)
• Can GPT-3.5 Generate and Code Discharge Summaries? [45.633849969788315]
  We generated and coded 9,606 discharge summaries based on lists of ICD-10 code descriptions. Neural coding models were trained on baseline and augmented data. We report micro- and macro-F1 scores on the full codeset, generation codes, and their families.
  arXiv  Detail & Related papers  (2024-01-24T15:10:13Z)
• Automated clinical coding using off-the-shelf large language models [10.365958121087305]
  The task of assigning diagnostic ICD codes to patient hospital admissions is typically performed by expert human coders. Efforts towards automated ICD coding are dominated by supervised deep learning models. In this work, we leverage off-the-shelf pre-trained generative large language models to develop a practical solution.
  arXiv  Detail & Related papers  (2023-10-10T11:56:48Z)
• Automated Medical Coding on MIMIC-III and MIMIC-IV: A Critical Review and Replicability Study [60.56194508762205]
  We reproduce, compare, and analyze state-of-the-art automated medical coding machine learning models. We show that several models underperform due to weak configurations, poorly sampled train-test splits, and insufficient evaluation. We present the first comprehensive results on the newly released MIMIC-IV dataset using the reproduced models.
  arXiv  Detail & Related papers  (2023-04-21T11:54:44Z)
• Read, Attend, and Code: Pushing the Limits of Medical Codes Prediction from Clinical Notes by Machines [0.42641920138420947]
  We present our Read, Attend, and Code (RAC) model for learning the medical code assignment mappings. RAC establishes a new state of the art (SOTA) considerably outperforming the current best Macro-F1 by 18.7%. This new milestone marks a meaningful step toward fully autonomous medical coding (AMC) in machines.
  arXiv  Detail & Related papers  (2021-07-10T06:01:58Z)
• A Meta-embedding-based Ensemble Approach for ICD Coding Prediction [64.42386426730695]
  International Classification of Diseases (ICD) are the de facto codes used globally for clinical coding. These codes enable healthcare providers to claim reimbursement and facilitate efficient storage and retrieval of diagnostic information. Our proposed approach enhances the performance of neural models by effectively training word vectors using routine medical data as well as external knowledge from scientific articles.
  arXiv  Detail & Related papers  (2021-02-26T17:49:58Z)
• Collaborative residual learners for automatic icd10 prediction using prescribed medications [45.82374977939355]
  We propose a novel collaborative residual learning based model to automatically predict ICD10 codes employing only prescriptions data. We obtain multi-label classification accuracy of 0.71 and 0.57 of average precision, 0.57 and 0.38 of F1-score and 0.73 and 0.44 of accuracy in predicting principal diagnosis for inpatient and outpatient datasets respectively.
  arXiv  Detail & Related papers  (2020-12-16T07:07:27Z)
• Ensemble model for pre-discharge icd10 coding prediction [45.82374977939355]
  We propose an ensemble model incorporating multiple clinical data sources for accurate code predictions. We obtain multi-label classification accuracies of 0.73 and 0.58 for average precision, 0.56 and 0.35 for F1-scores and 0.71 and 0.4 accuracy in predicting principal diagnosis for inpatient and outpatient datasets respectively.
  arXiv  Detail & Related papers  (2020-12-16T07:02:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.