Agentic AI-based Framework for Mitigating Premature Diagnostic Handoff and Silent Hallucination in Healthcare Applications

• URL: http://arxiv.org/abs/2606.18068v1
• Date: Tue, 16 Jun 2026 15:39:19 GMT
• Title: Agentic AI-based Framework for Mitigating Premature Diagnostic Handoff and Silent Hallucination in Healthcare Applications
• Abstract summary: Open-ended conversational agents remain prone to two critical failure modes: premature diagnostic handoff and silent clinical hallucinations.<n>We propose a multi-agent framework that addresses both issues by replacing LLM-as-a-judge'' routing with deterministic orchestration constraints.<n>We evaluate the system using simulated patient agents powered by the llama-3.1-70b-instruct model on 150 test cases.
• Score: 17.69274632062373
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in Large Language Models (LLMs) and multi-agent systems have driven the rise of Agentic AI, showing promise for medical reasoning. However, open-ended conversational agents remain prone to two critical failure modes: premature diagnostic handoff and silent clinical hallucinations that may go undetected before reaching the patient. In this work, we propose a multi-agent framework that addresses both issues by replacing ``LLM-as-a-judge'' routing with deterministic orchestration constraints. The framework incorporates two safety mechanisms. First, a neuro-symbolic state-tracking gate enforces completeness of the OLDCARTS clinical protocol (Onset, Location, Duration, Character, Aggravating/Alleviating factors, Radiation, Timing, and Severity) by blocking diagnostic transitions until all required dimensions are collected. Second, an epistemic uncertainty quantification (UQ) gate computes semantic entropy (H) across K=5 independent diagnostic samples to identify and intercept divergent outputs before delivery. We evaluate the system using simulated patient agents powered by the llama-3.1-70b-instruct model on 150 test cases. The full architecture achieves 49.3% diagnostic precision, representing an absolute improvement of 11.3 percentage points over an unconstrained baseline. Additionally, we observe a statistically significant negative correlation (r = -0.181, p < 0.05) between OLDCARTS completeness (σ) and semantic entropy (H), suggesting that structured information gathering is associated with reduced diagnostic uncertainty.

Related papers

• Beyond Information Seeking: Severity-Aware Question Supervision for Proactive Medical Dialogue [3.62175729900672]
  We propose Expected-Severity-Risk (ESR), a consequence-aware question-supervision objective that values each candidate by its expected reduction in severity-aware terminal risk.<n>We show that matched ESR supervision reduces mean high-severity diagnostic miss from.0645 to.0455 (-29.5%) and improves mean diagnostic accuracy from.9123 to.9320 while requiring only 0.14 additional questions per dialogue.
  arXiv  Detail & Related papers  (2026-08-25T13:09:45Z)
• COTCAgent: Preventive Consultation via Probabilistic Chain-of-Thought Completion [1.3891530345631953]
  The Probabilistic Chain-of-Thought Completion Agent (COTCAgent) is a hierarchical reasoning framework for longitudinal electronic health records.<n>COTCAgent achieves 90.47% Top-1 accuracy on the self-built dataset and 70.41% on HealthBench, outperforming existing medical agents and mainstream large language models.
  arXiv  Detail & Related papers  (2026-05-14T16:17:35Z)
• ADAPTS: Agentic Decomposition for Automated Protocol-agnostic Tracking of Symptoms [36.94429692322632]
  ADAPTS (Agentic Decomposition for Automated Protocol-agnostic Tracking of Symptoms) is a framework for automated rating of depression and anxiety severity.<n>This approach decomposes long-form clinical interviews into symptom-specific reasoning tasks.<n>Ratings approximated expert benchmarks more closely than original human ratings.
  arXiv  Detail & Related papers  (2026-05-04T23:08:42Z)
• SAAG: Structured Agent Assessment and Grounding [13.62148061055744]
  We propose a cascaded diagnostic framework that decomposes agent-calling evaluation into three sequential stages.<n>We evaluate this framework on a benchmark derived from Glaive's function-calling dataset across registry sizes of 5, 10, and 15 agents.
  arXiv  Detail & Related papers  (2026-04-30T19:33:58Z)
• Benchmarking Multi-turn Medical Diagnosis: Hold, Lure, and Self-Correction [72.89352076103889]
  Large language models (LLMs) achieve high accuracy in medical diagnosis when all clinical information is provided in a single turn.<n>We introduce MINT, a high-fidelity, multi-turn medical diagnosis benchmark comprising 1,035 cases.<n>We uncover three persistent behavioral patterns that significantly impact diagnostic decisions.
  arXiv  Detail & Related papers  (2026-04-06T00:23:10Z)
• Agentic Cognitive Profiling: Realigning Automated Alzheimer's Disease Detection with Clinical Construct Validity [66.94391219005291]
  This paper proposes Agentic Cognitive Profiling (ACP), an agentic framework that realigns automated screening with clinical protocol logic.<n>Central to our design is decoupling semantic understanding from measurement by delegating all quantification to deterministic function calling.<n>ACP achieves 90.5% score match rate in task examination and 85.3% accuracy in AD prediction, surpassing popular baselines.
  arXiv  Detail & Related papers  (2026-03-18T06:15:35Z)
• Automated Self-Testing as a Quality Gate: Evidence-Driven Release Management for LLM Applications [51.56484100374058]
  We present an automated self-testing framework that introduces quality gates with evidence-based release decisions.<n>We evaluate the framework through a longitudinal case study of an internally deployed multi-agent conversational AI system.
  arXiv  Detail & Related papers  (2026-03-13T20:44:15Z)
• A Multi-Agent Framework for Medical AI: Leveraging Fine-Tuned GPT, LLaMA, and DeepSeek R1 for Evidence-Based and Bias-Aware Clinical Query Processing [0.4349324020366305]
  Large language models (LLMs) show promise for healthcare question answering, but clinical use is limited by weak verification, insufficient evidence grounding, and unreliable confidence signalling.<n>We propose a multi-agent medical QA framework that combines complementary LLMs with evidence retrieval, uncertainty estimation, and bias checks to improve answer reliability.
  arXiv  Detail & Related papers  (2026-02-15T14:17:27Z)
• MedAD-R1: Eliciting Consistent Reasoning in Interpretible Medical Anomaly Detection via Consistency-Reinforced Policy Optimization [46.65200216642429]
  We introduce MedAD-38K, the first large-scale, multi-modal, and multi-center benchmark for MedAD featuring diagnostic Chain-of-Thought (CoT) annotations alongside structured Visual Question-Answering (VQA) pairs.<n>Our proposed model, MedAD-R1, achieves state-of-the-art (SOTA) performance on the MedAD-38K benchmark, outperforming strong baselines by more than 10%.
  arXiv  Detail & Related papers  (2026-02-01T07:56:10Z)
• An Agentic System for Rare Disease Diagnosis with Traceable Reasoning [69.46279475491164]
  We introduce DeepRare, the first rare disease diagnosis agentic system powered by a large language model (LLM)<n>DeepRare generates ranked diagnostic hypotheses for rare diseases, each accompanied by a transparent chain of reasoning.<n>The system demonstrates exceptional diagnostic performance among 2,919 diseases, achieving 100% accuracy for 1013 diseases.
  arXiv  Detail & Related papers  (2025-06-25T13:42:26Z)
• MEDDxAgent: A Unified Modular Agent Framework for Explainable Automatic Differential Diagnosis [17.888920170796457]
  Differential Diagnosis (DDx) is a fundamental yet complex aspect of clinical decision-making.<n>Recent advances in large language models (LLMs) have shown promise in supporting DDx.<n>We introduce a Modular Explainable DDx Agent (MEDDxAgent) framework designed for interactive DDx.
  arXiv  Detail & Related papers  (2025-02-26T14:31:43Z)
• Towards Causality-Aware Inferring: A Sequential Discriminative Approach for Medical Diagnosis [142.90770786804507]
  Medical diagnosis assistant (MDA) aims to build an interactive diagnostic agent to sequentially inquire about symptoms for discriminating diseases. This work attempts to address these critical issues in MDA by taking advantage of the causal diagram. We propose a propensity-based patient simulator to effectively answer unrecorded inquiry by drawing knowledge from the other records.
  arXiv  Detail & Related papers  (2020-03-14T02:05:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.