Closed-Loop Validation-Repair for Healthcare Interoperability: A Multi-Model Study of Schema Compliance in Clinical LLMs

• URL: http://arxiv.org/abs/2607.24371v1
• Date: Mon, 27 Jul 2026 12:45:27 GMT
• Title: Closed-Loop Validation-Repair for Healthcare Interoperability: A Multi-Model Study of Schema Compliance in Clinical LLMs
• Abstract summary: We evaluate three open-source models, Qwen2.5 7B, Llama 3.1 8B, and Gemma2 9B, via local deployment across 320 clinical scenarios spanning ten medical specialties.<n> schema noncompliance is consistent across the three model families, with baseline compliance rates ranging from 85.9 to 91.6 percent.<n> validation-repair framework achieves 99.0 percent overall compliance, ranging from 98.4 to 99.4 percent across models.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Healthcare interoperability requires AI systems to produce structured outputs conforming to standardized schemas including ICD-10 for diagnostic coding, CPT for procedure billing, and HL7 FHIR for data exchange. While large language models demonstrate clinical reasoning capabilities, their integration into electronic health record systems faces a critical barrier: schema noncompliance. We evaluate three open-source models, Qwen2.5 7B, Llama 3.1 8B, and Gemma2 9B, via local deployment across 320 clinical scenarios spanning ten medical specialties, yielding 960 model-scenario pairs assessed under paired baseline and validation-repair conditions. First, schema noncompliance is consistent across the three model families, with baseline compliance rates ranging from 85.9 to 91.6 percent despite varying architectures and training data, suggesting shared gaps in medical training corpora rather than model-specific limitations. Second, 96 percent of validator-detected failures are representation-level format violations such as alternative medical abbreviations and code prefixes, indicating models follow clinical writing conventions but lack awareness of healthcare IT standards. Third, the validation-repair framework achieves 99.0 percent overall compliance, ranging from 98.4 to 99.4 percent across models, with most errors resolving within one or two iterations. Exact McNemar p-values below 0.001 and absolute improvements of 7.8 to 12.5 percentage points across model sizes confirm statistical significance. These results support closed-loop validation-repair as an effective system-level safeguard for healthcare interoperability, improving schema-level readiness for downstream clinical system integration.

Related papers

• IyawoBench v2.0: Extended Diagnostic Evaluation of Large Language Model Clinical Triage in Nigerian Primary Care [0.0]
  We present IyawoBench v2.0, an extended diagnostic evaluation of large language model clinical triage.<n>We introduce a formal mathematical framework that decomposes triage safety into three distinct failure modes: Conservative Escalation Bias, Systematic Downgrade Bias, and Middle-Tier Instability.
  arXiv  Detail & Related papers  (2026-07-31T07:11:55Z)
• MedPRMBench: A Fine-grained Benchmark for Process Reward Models in Medical Reasoning [7.000170880015254]
  We propose MedPRMBench, the first process-level reward model benchmark for the medical domain.<n>It is built through a three-phase pipeline based on Clinical Reasoning Blueprints (CRBs)<n>The benchmark comprises 6,500 questions with 13,000 reasoning chains and 113,910 step-level labels, plus 6,879 questions for training.
  arXiv  Detail & Related papers  (2026-04-19T06:44:07Z)
• HiPath: Hierarchical Vision-Language Alignment for Structured Pathology Report Prediction [5.151955122293638]
  HiPath is a lightweight pathology vision-language framework built on frozen UNI2 and Qwen3 backbones.<n>Trained on 749K real-world Chinese pathology cases, HiPath achieves 68.9% strict and 74.7% clinically acceptable accuracy with a 97.3% safety rate.
  arXiv  Detail & Related papers  (2026-03-20T13:58:02Z)
• A Federated and Parameter-Efficient Framework for Large Language Model Training in Medicine [59.78991974851707]
  Large language models (LLMs) have demonstrated strong performance on medical benchmarks, including question answering and diagnosis.<n>Most medical LLMs are trained on data from a single institution, which faces limitations in generalizability and safety in heterogeneous systems.<n>We introduce the model-agnostic and parameter-efficient federated learning framework for adapting LLMs to medical applications.
  arXiv  Detail & Related papers  (2026-01-29T18:48:21Z)
• From Performance to Practice: Knowledge-Distilled Segmentator for On-Premises Clinical Workflows [8.438143117044795]
  We present a deployment-oriented framework to translate a high-performing segmentation model into a scalable family of compact student models.<n>The proposed approach preserves architectural compatibility with existing clinical systems while enabling systematic capacity reduction.<n>Results show that knowledge distillation provides a practical and reliable pathway for converting research-grade segmentation models into maintainable, deployment-ready components.
  arXiv  Detail & Related papers  (2026-01-14T05:44:30Z)
• Hybrid-Code: A Privacy-Preserving, Redundant Multi-Agent Framework for Reliable Local Clinical Coding [0.0]
  Clinical coding automation using cloud-based Large Language Models (LLMs) poses privacy risks and latency bottlenecks.<n>We introduce Hybrid-Code, a hybrid neuro-symbolic multi-agent framework for local clinical coding that ensures production reliability through redundancy and verification.
  arXiv  Detail & Related papers  (2025-12-26T02:27:36Z)
• Colorectal Cancer Histopathological Grading using Multi-Scale Federated Learning [0.0]
  We propose a scalable, privacy-preserving federated learning framework for colorectal cancer grading.<n>Our framework achieves an overall accuracy of 83.5%, outperforming a comparable centralized model.<n>The proposed modular pipeline establishes a foundational step toward deployable, privacy-aware clinical AI for digital pathology.
  arXiv  Detail & Related papers  (2025-11-05T18:18:09Z)
• Med-RewardBench: Benchmarking Reward Models and Judges for Medical Multimodal Large Language Models [57.73472878679636]
  We introduce Med-RewardBench, the first benchmark specifically designed to evaluate medical reward models and judges.<n>Med-RewardBench features a multimodal dataset spanning 13 organ systems and 8 clinical departments, with 1,026 expert-annotated cases.<n>A rigorous three-step process ensures high-quality evaluation data across six clinically critical dimensions.
  arXiv  Detail & Related papers  (2025-08-29T08:58:39Z)
• Uncertainty-Driven Expert Control: Enhancing the Reliability of Medical Vision-Language Models [52.2001050216955]
  Existing methods aim to enhance the performance of Medical Vision Language Model (MedVLM) by adjusting model structure, fine-tuning with high-quality data, or through preference fine-tuning.<n>We propose an expert-in-the-loop framework named Expert-Controlled-Free Guidance (Expert-CFG) to align MedVLM with clinical expertise without additional training.
  arXiv  Detail & Related papers  (2025-07-12T09:03:30Z)
• An Agentic System for Rare Disease Diagnosis with Traceable Reasoning [69.46279475491164]
  We introduce DeepRare, the first rare disease diagnosis agentic system powered by a large language model (LLM)<n>DeepRare generates ranked diagnostic hypotheses for rare diseases, each accompanied by a transparent chain of reasoning.<n>The system demonstrates exceptional diagnostic performance among 2,919 diseases, achieving 100% accuracy for 1013 diseases.
  arXiv  Detail & Related papers  (2025-06-25T13:42:26Z)
• Towards Reliable Medical Image Segmentation by Modeling Evidential Calibrated Uncertainty [57.023423137202485]
  Concerns regarding the reliability of medical image segmentation persist among clinicians.<n>We introduce DEviS, an easily implementable foundational model that seamlessly integrates into various medical image segmentation networks.<n>By leveraging subjective logic theory, we explicitly model probability and uncertainty for medical image segmentation.
  arXiv  Detail & Related papers  (2023-01-01T05:02:46Z)
• Advancing COVID-19 Diagnosis with Privacy-Preserving Collaboration in Artificial Intelligence [79.038671794961]
  We launch the Unified CT-COVID AI Diagnostic Initiative (UCADI), where the AI model can be distributedly trained and independently executed at each host institution. Our study is based on 9,573 chest computed tomography scans (CTs) from 3,336 patients collected from 23 hospitals located in China and the UK.
  arXiv  Detail & Related papers  (2021-11-18T00:43:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.