LLM-Orchestrated Conformance Checking in Stroke Care Without Computer-Interpretable Guidelines

• URL: http://arxiv.org/abs/2606.09489v1
• Date: Mon, 08 Jun 2026 13:44:03 GMT
• Title: LLM-Orchestrated Conformance Checking in Stroke Care Without Computer-Interpretable Guidelines
• Abstract summary: Conformance checking in healthcare seeks to assess whether patient care pathways adhere to clinical guidelines.<n>This work introduces a modular framework to support medical conformance checking directly from unstructured clinical and guideline texts.
• Score: 35.18016233072556
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Objective: Conformance checking in healthcare seeks to assess whether patient care pathways adhere to clinical guidelines. However, its practical application often depends on the availability of formal, machine-interpretable representations of guidelines, such as Computer-Interpretable Guidelines (CIGs), which are seldom available in real-world clinical settings. Methods: This work introduces a modular framework based on the orchestration of Large Language Models (LLMs) to support medical conformance checking directly from unstructured clinical and guideline texts, without requiring predefined CIGs. The proposed architecture integrates multiple LLMs and supporting components to extract patient traces from clinical discharge letters, identify normative rules from textual clinical guidelines, translate these rules into executable scripts, and compute a Trace Conformance Indicator to quantify compliance within the event log. Results: The framework was implemented and evaluated in the stroke care domain at the neurological ward of Alessandria Hospital. Hundreds of patient traces were automatically extracted from hospital data and assessed against 50 rules derived from the reference guideline. The analysis showed that more than 86\% of the available traces were conformant. Conclusion: The results demonstrate the feasibility of using orchestrated LLMs for practical healthcare conformance analysis. At the same time, the study provides evidence of a high level of adherence to stroke care guidelines at Alessandria Hospital.

Related papers

• Benchmarking Clinical Decision Pathway Adherence in Large Language Models [111.31765633748081]
  MEGA-CDP is a benchmark for evaluating whether medical LLMs can generate guideline-adherent CDPs.<n>It is constructed from 2,274 English and Chinese clinical practice guidelines through a guideline-to-case pipeline.<n>It supports both single-turn vignette and multi-turn interactive settings, and introduces a CDP-oriented evaluation framework.
  arXiv  Detail & Related papers  (2026-08-27T04:17:12Z)
• Evaluating Large Language Models in Dynamic Clinical Decision-Making with Standardized Patient Cases [71.12461204050985]
  MedSP1000 is an SP-derived interactive benchmark for clinical-agent evaluation.<n>It converts peer-reviewed SP teaching cases into executable scenarios with defined SP case scripts, clinical environment contexts, and human-validated structured rubrics.<n>Applying MedSP1000 to a range of general-purpose and medically specialized LLMs, we find that performance on static benchmarks does not reliably translate to such educational scenarios.
  arXiv  Detail & Related papers  (2026-06-03T17:17:16Z)
• MedCase-Structured: A Text-to-FHIR Dataset for Benchmarking Diagnostic Reasoning in Clinically Realistic EHR Settings [2.759077490183459]
  Large language models (LLMs) show promise for clinical reasoning and decision support.<n>We introduce a pipeline for generating clinically realistic HL7 FHIR R4 bundles from unstructured text.<n>Applying this approach to MedCaseReasoning, we construct MedCase-Structured, a synthetic dataset aligned with clinician-authored diagnostic cases.
  arXiv  Detail & Related papers  (2026-05-28T17:42:43Z)
• MED-COPILOT: A Medical Assistant Powered by GraphRAG and Similar Patient Case Retrieval [12.265116154395434]
  We present MED-COPILOT, an interactive clinical decision-support system designed for clinicians and medical trainees.<n>The system builds a structured knowledge graph from WHO and NICE guidelines, applies community-level summarization for efficient retrieval, and maintains a 36,000-case similar-patient database.
  arXiv  Detail & Related papers  (2026-02-28T04:32:03Z)
• ClinAlign: Scaling Healthcare Alignment from Clinician Preference [46.61989657665566]
  We introduce HealthRubrics, a dataset of 7,034 physician-verified preference examples.<n>We distill these rubrics into HealthPrinciples: 119 broadly reusable, clinically grounded principles.<n>A 30B-A3B model trained with our framework achieves 33.4% on HealthBench-Hard.
  arXiv  Detail & Related papers  (2026-02-10T11:02:57Z)
• AgentsEval: Clinically Faithful Evaluation of Medical Imaging Reports via Multi-Agent Reasoning [73.50200033931148]
  We introduce AgentsEval, a multi-agent stream reasoning framework that emulates the collaborative diagnostic workflow of radiologists.<n>By dividing the evaluation process into interpretable steps including criteria definition, evidence extraction, alignment, and consistency scoring, AgentsEval provides explicit reasoning traces and structured clinical feedback.<n> Experimental results demonstrate that AgentsEval delivers clinically aligned, semantically faithful, and interpretable evaluations that remain robust under paraphrastic, semantic, and stylistic perturbations.
  arXiv  Detail & Related papers  (2026-01-23T11:59:13Z)
• ClinDEF: A Dynamic Evaluation Framework for Large Language Models in Clinical Reasoning [58.01333341218153]
  We propose ClinDEF, a dynamic framework for assessing clinical reasoning in LLMs through simulated diagnostic dialogues.<n>Our method generates patient cases and facilitates multi-turn interactions between an LLM-based doctor and an automated patient agent.<n>Experiments show that ClinDEF effectively exposes critical clinical reasoning gaps in state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2025-12-29T12:58:58Z)
• Leveraging LLMs for Structured Data Extraction from Unstructured Patient Records [0.0]
  Manual chart review remains an extremely time-consuming and resource-intensive component of clinical research.<n>We present a framework for automated structured feature extraction from clinical notes leveraging locally deployed large language models (LLMs)<n>This framework demonstrates the potential of LLM systems to reduce the burden of manual chart review and increase consistency in data capture.
  arXiv  Detail & Related papers  (2025-12-03T14:10:12Z)
• Refine Medical Diagnosis Using Generation Augmented Retrieval and Clinical Practice Guidelines [16.56254046507092]
  We introduce GARMLE-G, a Generation-Augmented Retrieval framework that grounds medical language model outputs in authoritative guidelines.<n>Unlike conventional Retrieval-Augmented Generation based approaches, GARMLE-G enables hallucination-free outputs by directly retrieving authoritative guideline content.<n>A prototype system for hypertension diagnosis was developed and evaluated on multiple metrics, demonstrating superior retrieval precision, semantic relevance, and clinical guideline adherence.
  arXiv  Detail & Related papers  (2025-06-22T11:31:13Z)
• Bringing CLIP to the Clinic: Dynamic Soft Labels and Negation-Aware Learning for Medical Analysis [0.9944647907864256]
  We propose a novel approach that integrates clinically-enhanced dynamic soft labels and medical graphical alignment.<n>Our approach is easily integrated into the medical CLIP training pipeline and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-05-28T08:00:18Z)
• Self-supervised Answer Retrieval on Clinical Notes [68.87777592015402]
  We introduce CAPR, a rule-based self-supervision objective for training Transformer language models for domain-specific passage matching. We apply our objective in four Transformer-based architectures: Contextual Document Vectors, Bi-, Poly- and Cross-encoders. We report that CAPR outperforms strong baselines in the retrieval of domain-specific passages and effectively generalizes across rule-based and human-labeled passages.
  arXiv  Detail & Related papers  (2021-08-02T10:42:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.