Retrieval-Augmented Guardrails for AI-Drafted Patient-Portal Messages: Error Taxonomy Construction and Large-Scale Evaluation

• URL: http://arxiv.org/abs/2509.22565v1
• Date: Fri, 26 Sep 2025 16:42:43 GMT
• Title: Retrieval-Augmented Guardrails for AI-Drafted Patient-Portal Messages: Error Taxonomy Construction and Large-Scale Evaluation
• Authors: Wenyuan Chen, Fateme Nateghi Haredasht, Kameron C. Black, Francois Grolleau, Emily Alsentzer, Jonathan H. Chen, Stephen P. Ma,
• Abstract summary: Asynchronous patient-clinician messaging via EHR portals is a growing source of clinician workload.<n>Our contributions are threefold: (1) we introduce a clinically grounded error ontology comprising 5 domains and 59 granular error codes; (2) we develop a retrieval-augmented evaluation pipeline; and (3) we provide a two-stage prompting architecture using DSPy to enable scalable, interpretable, and hierarchical error detection.
• Score: 5.555479009357263
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Asynchronous patient-clinician messaging via EHR portals is a growing source of clinician workload, prompting interest in large language models (LLMs) to assist with draft responses. However, LLM outputs may contain clinical inaccuracies, omissions, or tone mismatches, making robust evaluation essential. Our contributions are threefold: (1) we introduce a clinically grounded error ontology comprising 5 domains and 59 granular error codes, developed through inductive coding and expert adjudication; (2) we develop a retrieval-augmented evaluation pipeline (RAEC) that leverages semantically similar historical message-response pairs to improve judgment quality; and (3) we provide a two-stage prompting architecture using DSPy to enable scalable, interpretable, and hierarchical error detection. Our approach assesses the quality of drafts both in isolation and with reference to similar past message-response pairs retrieved from institutional archives. Using a two-stage DSPy pipeline, we compared baseline and reference-enhanced evaluations on over 1,500 patient messages. Retrieval context improved error identification in domains such as clinical completeness and workflow appropriateness. Human validation on 100 messages demonstrated superior agreement (concordance = 50% vs. 33%) and performance (F1 = 0.500 vs. 0.256) of context-enhanced labels vs. baseline, supporting the use of our RAEC pipeline as AI guardrails for patient messaging.

Related papers

• An Agentic AI System for Multi-Framework Communication Coding [17.846847341760675]
  We developed a Multi-framework Structured Agentic AI system for Clinical Communication (MOSAIC)<n>MOSAIC is built on a LangGraph-based architecture that orchestrates four core agents, including a Plan Agent for codebook selection and workflow planning.<n>To evaluate performance, we compared MOSAIC outputs against gold-standard annotations created by trained human coders.
  arXiv  Detail & Related papers  (2025-12-09T14:46:16Z)
• WER is Unaware: Assessing How ASR Errors Distort Clinical Understanding in Patient Facing Dialogue [3.468314243424983]
  Automatic Speech Recognition (ASR) is increasingly deployed in clinical dialogue.<n>Standard evaluations still rely heavily on Error Error Rate (WER)<n>This paper challenges that standard, investigating whether WER or other common metrics correlate with clinical impact of transcription errors.
  arXiv  Detail & Related papers  (2025-11-20T16:59:20Z)
• DispatchMAS: Fusing taxonomy and artificial intelligence agents for emergency medical services [49.70819009392778]
  Large Language Models (LLMs) and Multi-Agent Systems (MAS) offer opportunities to augment dispatchers.<n>This study aimed to develop and evaluate a taxonomy-grounded, multi-agent system for simulating realistic scenarios.
  arXiv  Detail & Related papers  (2025-10-24T08:01:21Z)
• MedRepBench: A Comprehensive Benchmark for Medical Report Interpretation [2.3251933592942247]
  We introduce MedRepBench, a comprehensive benchmark built from 1,900 de-identified real-world Chinese medical reports.<n>The benchmark is designed primarily to evaluate end-to-end VLMs for structured medical report understanding.<n>We also observe that the OCR+LLM pipeline, despite strong performance, suffers from layout-blindness and latency issues.
  arXiv  Detail & Related papers  (2025-08-21T07:52:45Z)
• Development and Comparative Evaluation of Three Artificial Intelligence Models (NLP, LLM, JEPA) for Predicting Triage in Emergency Departments: A 7-Month Retrospective Proof-of-Concept [0.0]
  Emergency departments struggle with persistent triage errors, especially undertriage and overtriage.<n>This study evaluated three AI models [TRIAGEmaster (NLP), URGENTIAPARSE (LLM), and EMERGINET (JEPA)] against the FRENCH triage scale and nurse practice.
  arXiv  Detail & Related papers  (2025-07-01T16:37:55Z)
• A Dataset for Addressing Patient's Information Needs related to Clinical Course of Hospitalization [15.837772594006038]
  ArchEHR-QA is an expert-annotated dataset based on real-world patient cases from intensive care unit and emergency department settings.<n>Cases comprise questions posed by patients to public health forums, clinician-interpreted counterparts, relevant clinical note excerpts with sentence-level relevance annotations, and clinician-authored answers.<n>The answer-first prompting approach consistently performed best, with Llama 4 achieving the highest scores.
  arXiv  Detail & Related papers  (2025-06-04T16:55:08Z)
• Real-world validation of a multimodal LLM-powered pipeline for High-Accuracy Clinical Trial Patient Matching leveraging EHR data [0.0]
  Patient recruitment in clinical trials is hindered by complex eligibility criteria and labor-intensive chart reviews.<n>We introduce an integration-free, LLM-powered pipeline that automates patient-trial matching using unprocessed documents extracted from EHRs.<n>Our approach leverages (1) the new reasoning-LLM paradigm, enabling the assessment of even the most complex criteria, (2) visual capabilities of latest LLMs to interpret medical records without lossy image-to-text conversions, and (3) multimodal embeddings for efficient medical record search.
  arXiv  Detail & Related papers  (2025-03-19T16:12:11Z)
• Quantifying the Reasoning Abilities of LLMs on Real-world Clinical Cases [48.87360916431396]
  We introduce MedR-Bench, a benchmarking dataset of 1,453 structured patient cases, annotated with reasoning references.<n>We propose a framework encompassing three critical examination recommendation, diagnostic decision-making, and treatment planning, simulating the entire patient care journey.<n>Using this benchmark, we evaluate five state-of-the-art reasoning LLMs, including DeepSeek-R1, OpenAI-o3-mini, and Gemini-2.0-Flash Thinking, etc.
  arXiv  Detail & Related papers  (2025-03-06T18:35:39Z)
• LlaMADRS: Prompting Large Language Models for Interview-Based Depression Assessment [75.44934940580112]
  This study introduces LlaMADRS, a novel framework leveraging open-source Large Language Models (LLMs) to automate depression severity assessment.<n>We employ a zero-shot prompting strategy with carefully designed cues to guide the model in interpreting and scoring transcribed clinical interviews.<n>Our approach, tested on 236 real-world interviews, demonstrates strong correlations with clinician assessments.
  arXiv  Detail & Related papers  (2025-01-07T08:49:04Z)
• Reshaping Free-Text Radiology Notes Into Structured Reports With Generative Transformers [0.29530625605275984]
  structured reporting (SR) has been recommended by various medical societies. We propose a pipeline to extract information from free-text reports. Our work aims to leverage the potential of Natural Language Processing (NLP) and Transformer-based models.
  arXiv  Detail & Related papers  (2024-03-27T18:38:39Z)
• Self-supervised Answer Retrieval on Clinical Notes [68.87777592015402]
  We introduce CAPR, a rule-based self-supervision objective for training Transformer language models for domain-specific passage matching. We apply our objective in four Transformer-based architectures: Contextual Document Vectors, Bi-, Poly- and Cross-encoders. We report that CAPR outperforms strong baselines in the retrieval of domain-specific passages and effectively generalizes across rule-based and human-labeled passages.
  arXiv  Detail & Related papers  (2021-08-02T10:42:52Z)
• Text Mining to Identify and Extract Novel Disease Treatments From Unstructured Datasets [56.38623317907416]
  We use Google Cloud to transcribe podcast episodes of an NPR radio show. We then build a pipeline for systematically pre-processing the text. Our model successfully identified that Omeprazole can help treat heartburn.
  arXiv  Detail & Related papers  (2020-10-22T19:52:49Z)
• COMPOSE: Cross-Modal Pseudo-Siamese Network for Patient Trial Matching [70.08786840301435]
  We propose CrOss-Modal PseudO-SiamEse network (COMPOSE) to address these challenges for patient-trial matching. Experiment results show COMPOSE can reach 98.0% AUC on patient-criteria matching and 83.7% accuracy on patient-trial matching.
  arXiv  Detail & Related papers  (2020-06-15T21:01:33Z)
• DeepEnroll: Patient-Trial Matching with Deep Embedding and Entailment Prediction [67.91606509226132]
  Clinical trials are essential for drug development but often suffer from expensive, inaccurate and insufficient patient recruitment. DeepEnroll is a cross-modal inference learning model to jointly encode enrollment criteria (tabular data) into a shared latent space for matching inference.
  arXiv  Detail & Related papers  (2020-01-22T17:51:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.