When Patients Cut In: Extending Clinical Conversational AI Safety to Interruptions

• URL: http://arxiv.org/abs/2608.29241v1
• Date: Sat, 29 Aug 2026 12:54:50 GMT
• Title: When Patients Cut In: Extending Clinical Conversational AI Safety to Interruptions
• Abstract summary: Clinical voice agents are now deployed in routine care, where real patients do not wait their turn: they interrupt.<n>Clinical conversational-AI benchmarks almost universally assume patients wait for the agent to finish, missing interruption-induced loss of required content.<n>We present a transcript-based evaluation of interruption recovery, adapting conversation-analytic overlap categories into three operational types.
• Score: 1.759008116536278
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Clinical voice agents are now deployed in routine care, where real patients do not wait their turn: they interrupt. These systems typically use a cascaded architecture (speech-to-text -> LLM -> text-to-speech), so when a patient cuts the agent off mid-utterance, clinically required content can be lost even when the model handles cooperative transcripts well. Yet clinical conversational-AI benchmarks almost universally assume patients wait for the agent to finish, missing interruption-induced loss of required content. We present a transcript-based evaluation of interruption recovery, adapting conversation-analytic overlap categories into three operational types (recognitional, competitive, transitional sub-unit) and testing four deployment-oriented, non-reasoning LLM configurations across four cells spanning history-taking (information gathering) and FAQ (information provision), scored on whether the agent preserves the clinically required content. In the gathering cells, target-question failure varied across models; in the provision cells, where arms are directly comparable, failure rose for every model. Rankings differ across cells, and competitive FAQ interruption produced 30/30 provision-coverage failures for all four models (Wilson 95% CI: 88.6-100.0%; baseline 0/30 for three, 4/30 for Llama). A brief apology marker ("sorry to interrupt") shifts recovery by tens of percentage points, inconsistently across models, and for one it reduces recovery. Interruption robustness therefore cannot be a single score: evaluation must be content-grounded, reported per cell, and matched to the deployment's interruption profile.

Related papers

• PatientAgentBench: A Benchmark Framework for Evaluating Patient-Facing Health AI Agents [5.247635961504091]
  PatientAgentBench benchmarks patient-facing agentic healthcare.<n>It evaluates a foundation model wrapped in an agent with a sandbox of healthcare tools, conversing with a simulated patient.<n>We benchmarked 10 models across four families on the same 1,200 scenarios and found clinical gaps.
  arXiv  Detail & Related papers  (2026-07-28T09:24:04Z)
• Agentic AI-based Framework for Mitigating Premature Diagnostic Handoff and Silent Hallucination in Healthcare Applications [17.69274632062373]
  Open-ended conversational agents remain prone to two critical failure modes: premature diagnostic handoff and silent clinical hallucinations.<n>We propose a multi-agent framework that addresses both issues by replacing LLM-as-a-judge'' routing with deterministic orchestration constraints.<n>We evaluate the system using simulated patient agents powered by the llama-3.1-70b-instruct model on 150 test cases.
  arXiv  Detail & Related papers  (2026-06-16T15:39:19Z)
• Evaluating Large Language Models in Dynamic Clinical Decision-Making with Standardized Patient Cases [71.12461204050985]
  MedSP1000 is an SP-derived interactive benchmark for clinical-agent evaluation.<n>It converts peer-reviewed SP teaching cases into executable scenarios with defined SP case scripts, clinical environment contexts, and human-validated structured rubrics.<n>Applying MedSP1000 to a range of general-purpose and medically specialized LLMs, we find that performance on static benchmarks does not reliably translate to such educational scenarios.
  arXiv  Detail & Related papers  (2026-06-03T17:17:16Z)
• ClinEnv: An Interactive Multi-Stage Long Horizon EHR Environment for Agents [14.434996950038956]
  ClinEnv scores both what the model decides, through deterministic-grounded matching, and how it gathers information.<n> ClinEnv makes this information-acquisition gap, invisible to outcome-only evaluation, directly measurable.
  arXiv  Detail & Related papers  (2026-06-01T17:56:26Z)
• Detecting Clinical Discrepancies in Health Coaching Agents: A Dual-Stream Memory and Reconciliation Architecture [71.46525715889656]
  General-purpose agent memory systems optimize for coherence by overwriting older facts with the user's latest statement.<n>We introduce a Dual-Stream Memory Architecture that strictly separates the patient narrative from the structured clinical record.<n>We evaluate this architecture on 26 patients across 675 longitudinal wellness coaching sessions.
  arXiv  Detail & Related papers  (2026-04-29T17:59:28Z)
• Automating Clinical Information Retrieval from Finnish Electronic Health Records Using Large Language Models [0.0]
  Locally deployable framework answers clinical questions directly from EHRs without external data transfer.<n>Open-source large language models (LLMs) ranging from 4B to 70B parameters were benchmarked under fully offline conditions.
  arXiv  Detail & Related papers  (2026-03-27T14:03:51Z)
• Same Verdict, Different Reasons: LLM-as-a-Judge and Clinician Disagreement on Medical Chatbot Completeness [49.2667937337333]
  We stress-test this assumption for detecting incomplete patient-facing medical responses.<n>We evaluate three granularities (General-Likert, Analytical-Rubric, Dynamic-Checklist) and three backbone models across two clinician-annotated datasets.
  arXiv  Detail & Related papers  (2026-03-26T19:01:55Z)
• MedForget: Hierarchy-Aware Multimodal Unlearning Testbed for Medical AI [66.0701326117134]
  MedForget is a hierarchy-aware multimodal unlearning testbed for building compliant medical AI systems.<n>We show that existing methods struggle to achieve complete, hierarchy-aware forgetting without reducing diagnostic performance.<n>We introduce a reconstruction attack that progressively adds hierarchical level context to prompts.
  arXiv  Detail & Related papers  (2025-12-10T17:55:06Z)
• Conformal Lesion Segmentation for 3D Medical Images [82.92159832699583]
  We propose a risk-constrained framework that calibrates data-driven thresholds via conformalization to ensure the test-time FNR remains below a target tolerance.<n>We validate the statistical soundness and predictive performance of CLS on six 3D-LS datasets across five backbone models, and conclude with actionable insights for deploying risk-aware segmentation in clinical practice.
  arXiv  Detail & Related papers  (2025-10-19T08:21:00Z)
• MEDEQUALQA: Evaluating Biases in LLMs with Counterfactual Reasoning [7.167933033102407]
  We introduce MEDEQUALQA, a counterfactual benchmark that perturbs only patient pronouns while holding critical symptoms and conditions constant.<n>We evaluate a GPT-4.1 model and compute Semantic Textual Similarity (STS) between reasoning traces to measure stability across pronoun variants.<n>Our results show overall high similarity (mean STS >0.80), but reveal consistent localized divergences in cited risk factors, guideline anchors, and differential ordering.
  arXiv  Detail & Related papers  (2025-10-09T22:12:58Z)
• Automatically measuring speech fluency in people with aphasia: first achievements using read-speech data [55.84746218227712]
  This study aims at assessing the relevance of a signalprocessingalgorithm, initially developed in the field of language acquisition, for the automatic measurement of speech fluency.
  arXiv  Detail & Related papers  (2023-08-09T07:51:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.