Leveraging Open-Source Large Language Models for encoding Social Determinants of Health using an Intelligent Router

• URL: http://arxiv.org/abs/2405.19631v1
• Date: Thu, 30 May 2024 02:33:28 GMT
• Title: Leveraging Open-Source Large Language Models for encoding Social Determinants of Health using an Intelligent Router
• Abstract summary: Social Determinants of Health (SDOH) play a significant role in patient health outcomes. Large language models (LLMs) show promise on extracting unstructured data from EHRs. We introduce an intelligent routing system for SDOH coding that uses a language model router to direct medical record data to open source LLMs.
• Score: 0.7499722271664144
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Social Determinants of Health (SDOH) play a significant role in patient health outcomes. The Center of Disease Control (CDC) introduced a subset of ICD-10 codes called Z-codes in an attempt to officially recognize and measure SDOH in the health care system. However, these codes are rarely annotated in a patient's Electronic Health Record (EHR), and instead, in many cases, need to be inferred from clinical notes. Previous research has shown that large language models (LLMs) show promise on extracting unstructured data from EHRs. However, with thousands of models to choose from with unique architectures and training sets, it's difficult to choose one model that performs the best on coding tasks. Further, clinical notes contain trusted health information making the use of closed-source language models from commercial vendors difficult, so the identification of open source LLMs that can be run within health organizations and exhibits high performance on SDOH tasks is an urgent problem. Here, we introduce an intelligent routing system for SDOH coding that uses a language model router to direct medical record data to open source LLMs that demonstrate optimal performance on specific SDOH codes. The intelligent routing system exhibits state of the art performance of 97.4% accuracy averaged across 5 codes, including homelessness and food insecurity, on par with closed models such as GPT-4o. In order to train the routing system and validate models, we also introduce a synthetic data generation and validation paradigm to increase the scale of training data without needing privacy protected medical records. Together, we demonstrate an architecture for intelligent routing of inputs to task-optimal language models to achieve high performance across a set of medical coding sub-tasks.

Related papers

• Towards Autonomous and Auditable Medical Imaging Model Development [53.326510777116766]
  We introduce AMID, an autonomous multi-agent framework for medical imaging model development.<n>AMID first proposes Data-Conditioned Method Planning, which refines coarse task-level search spaces into executable, parallelizable method lanes.<n>It then develops Verification-Guided Two-Stage Optimization, moving from broad early exploration of diverse method lanes to selective exploitation of promising candidates.
  arXiv  Detail & Related papers  (2026-07-12T00:54:06Z)
• Eliciting Medical Reasoning with Knowledge-enhanced Data Synthesis: A Semi-Supervised Reinforcement Learning Approach [62.0906177191353]
  Existing approaches distill chain-of-thought reasoning traces from large proprietary models via supervised fine-tuning, then conduct reinforcement learning (RL)<n>We propose MedSSR, a Medical Knowledge-enhanced data Synthesis and Semi-supervised Reinforcement learning framework.<n>Our framework first employs rare disease knowledge to synthesize distribution-controllable reasoning questions.<n>We then utilize the policy model itself to generate high-quality pseudo-labels. This enables a two-stage, intrinsic-to-extrinsic training paradigm: self-supervised RL on the pseudo-labeled data, followed by supervised RL on the human-annotated real data
  arXiv  Detail & Related papers  (2026-04-13T14:37:38Z)
• Training a Large Language Model for Medical Coding Using Privacy-Preserving Synthetic Clinical Data [10.24013423253024]
  We investigate whether a modern open-weight foundation model can be adapted for an expert-level medical coding task.<n>We fine-tune Llama 3-70B on pairs of clinical notes and gold codes generated from EHR-grounded templates and coding policies.<n>After fine-tuning on the synthetic corpus, exact-match F1 exceeded 0.70, representing a large absolute gain across both code systems.
  arXiv  Detail & Related papers  (2026-03-06T02:02:49Z)
• Leveraging Large Language Models to Extract and Translate Medical Information in Doctors' Notes for Health Records and Diagnostic Billing Codes [0.0]
  This thesis explores using open-weight Large Language Models (LLMs) to extract clinical information from physician notes and translate it into ICD-10-CM diagnostic codes without reliance on cloud-based services.<n>A privacy-focused pipeline was developed using Ollama, LangChain, and containerized environments to evaluate multiple open-weight models.<n>Results show that strict schema enforcement achieved near 100% compliance, but accurate generation of specific diagnostic codes remains challenging for smaller models.
  arXiv  Detail & Related papers  (2026-01-14T12:21:12Z)
• ACE-ICD: Acronym Expansion As Data Augmentation For Automated ICD Coding [2.0267433264426598]
  We propose a novel effective data augmentation technique that leverages large language models to expand medical acronyms.<n>Our approach, ACE-ICD establishes new state-of-the-art performance across multiple settings, including common codes, rare codes, and full-code assignments.
  arXiv  Detail & Related papers  (2025-11-10T17:11:20Z)
• SciML Agents: Write the Solver, Not the Solution [69.5021018644143]
  We introduce two new datasets: a diagnostic dataset of adversarial "misleading" problems; and a large-scale benchmark of 1,000 diverse ODE tasks.<n>We evaluate open- and closed-source LLM models along two axes: (i) unguided versus guided prompting with domain-specific knowledge; and (ii) off-the-shelf versus fine-tuned variants.<n>Preliminary results indicate that careful prompting and fine-tuning can yield a specialized LLM agent capable of reliably solving simple ODE problems.
  arXiv  Detail & Related papers  (2025-09-12T02:53:57Z)
• RuCCoD: Towards Automated ICD Coding in Russian [42.609069328685045]
  This study investigates the feasibility of automating clinical coding in Russian, a language with limited biomedical resources.<n>We present a new dataset for ICD coding, which includes diagnosis fields from electronic health records annotated with over 10,000 entities and more than 1,500 unique ICD codes.
  arXiv  Detail & Related papers  (2025-02-28T17:40:24Z)
• Large Language Models are Powerful Electronic Health Record Encoders [2.310631440585048]
  General-purpose Large Language Models are used to encode EHR into representations for downstream clinical prediction tasks.<n>We show that our approach requires no institution-specific training and can incorporate any medical code with a text description.<n>An LLM-based model achieves superior performance for prediction of disease onset, hospitalization, and mortality, indicating robustness to population and coding shifts.
  arXiv  Detail & Related papers  (2025-02-24T18:30:36Z)
• MKE-Coder: Multi-Axial Knowledge with Evidence Verification in ICD Coding for Chinese EMRs [9.615982382826768]
  This paper introduces a novel framework called MKE-Coder: Multi-axial Knowledge with Evidence verification in ICD coding for Chinese EMRs. We identify candidate codes for the diagnosis and categorize each of them into knowledge under four coding axes. We retrieve corresponding clinical evidence from the comprehensive content of EMRs and filter credible evidence through a scoring model.
  arXiv  Detail & Related papers  (2025-02-19T08:08:53Z)
• OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models [70.72097493954067]
  Large language models (LLMs) for code have become indispensable in various domains, including code generation, reasoning tasks and agent systems. While open-access code LLMs are increasingly approaching the performance levels of proprietary models, high-quality code LLMs remain limited. We introduce OpenCoder, a top-tier code LLM that not only achieves performance comparable to leading models but also serves as an "open cookbook" for the research community.
  arXiv  Detail & Related papers  (2024-11-07T17:47:25Z)
• FEDMEKI: A Benchmark for Scaling Medical Foundation Models via Federated Knowledge Injection [83.54960238236548]
  FEDMEKI not only preserves data privacy but also enhances the capability of medical foundation models. FEDMEKI allows medical foundation models to learn from a broader spectrum of medical knowledge without direct data exposure.
  arXiv  Detail & Related papers  (2024-08-17T15:18:56Z)
• HealthGAT: Node Classifications in Electronic Health Records using Graph Attention Networks [2.2026317523029193]
  HealthGAT is a graph attention network framework that generates embeddings from EHR. Our model iteratively refines the embeddings for medical codes, resulting in improved EHR data analysis. Our model shows outstanding performance in node classification and downstream tasks such as predicting readmissions and diagnosis classifications.
  arXiv  Detail & Related papers  (2024-03-26T22:17:01Z)
• Large Language Model Distilling Medication Recommendation Model [61.89754499292561]
  We harness the powerful semantic comprehension and input-agnostic characteristics of Large Language Models (LLMs) Our research aims to transform existing medication recommendation methodologies using LLMs. To mitigate this, we have developed a feature-level knowledge distillation technique, which transfers the LLM's proficiency to a more compact model.
  arXiv  Detail & Related papers  (2024-02-05T08:25:22Z)
• Automatic Coding at Scale: Design and Deployment of a Nationwide System for Normalizing Referrals in the Chilean Public Healthcare System [0.0]
  We propose a two-step system for automatically coding diseases in referrals from the Chilean public healthcare system. Specifically, our model uses a state-of-the-art NER model for recognizing disease mentions and a search engine system based on for assigning the most relevant codes associated with these disease mentions. Our system obtained a MAP score of 0.63 for the subcategory level and 0.83 for the category level, close to the best-performing models in the literature.
  arXiv  Detail & Related papers  (2023-07-09T16:19:35Z)
• ICDBigBird: A Contextual Embedding Model for ICD Code Classification [71.58299917476195]
  Contextual word embedding models have achieved state-of-the-art results in multiple NLP tasks. ICDBigBird is a BigBird-based model which can integrate a Graph Convolutional Network (GCN) Our experiments on a real-world clinical dataset demonstrate the effectiveness of our BigBird-based model on the ICD classification task.
  arXiv  Detail & Related papers  (2022-04-21T20:59:56Z)
• Unifying Heterogenous Electronic Health Records Systems via Text-Based Code Embedding [7.3394352452936085]
  We introduce DescEmb, a code-agnostic description-based representation learning framework for predictive modeling on EHR. We tested our model's capacity on various experiments including prediction tasks, transfer learning and pooled learning.
  arXiv  Detail & Related papers  (2021-08-08T12:47:42Z)
• TransICD: Transformer Based Code-wise Attention Model for Explainable ICD Coding [5.273190477622007]
  International Classification of Disease (ICD) coding procedure has been shown to be effective and crucial to the billing system in medical sector. Currently, ICD codes are assigned to a clinical note manually which is likely to cause many errors. In this project, we apply a transformer-based architecture to capture the interdependence among the tokens of a document and then use a code-wise attention mechanism to learn code-specific representations of the entire document.
  arXiv  Detail & Related papers  (2021-03-28T05:34:32Z)
• A Meta-embedding-based Ensemble Approach for ICD Coding Prediction [64.42386426730695]
  International Classification of Diseases (ICD) are the de facto codes used globally for clinical coding. These codes enable healthcare providers to claim reimbursement and facilitate efficient storage and retrieval of diagnostic information. Our proposed approach enhances the performance of neural models by effectively training word vectors using routine medical data as well as external knowledge from scientific articles.
  arXiv  Detail & Related papers  (2021-02-26T17:49:58Z)
• Explainable Automated Coding of Clinical Notes using Hierarchical Label-wise Attention Networks and Label Embedding Initialisation [4.4036730220831535]
  Recent studies on deep learning for automated medical coding achieved promising performances. We propose a Hierarchical Label-wise Attention Network (HLAN), which aimed to interpret the model by quantifying importance (as attention weights) of words and sentences related to each of the labels. Secondly, we propose to enhance the major deep learning models with a label embedding (LE) initialisation approach, which learns a dense, continuous vector representation and then injects the representation into the final layers and the label-wise attention layers in the models.
  arXiv  Detail & Related papers  (2020-10-29T16:21:26Z)
• DeepEnroll: Patient-Trial Matching with Deep Embedding and Entailment Prediction [67.91606509226132]
  Clinical trials are essential for drug development but often suffer from expensive, inaccurate and insufficient patient recruitment. DeepEnroll is a cross-modal inference learning model to jointly encode enrollment criteria (tabular data) into a shared latent space for matching inference.
  arXiv  Detail & Related papers  (2020-01-22T17:51:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.