Agentic AI in medicine: architectures, applications, evaluation, and challenges for clinical translation

• URL: http://arxiv.org/abs/2607.25489v1
• Date: Tue, 28 Jul 2026 09:25:25 GMT
• Title: Agentic AI in medicine: architectures, applications, evaluation, and challenges for clinical translation
• Abstract summary: The scope of agentic AI in medicine remains unsettled, and current evaluation practices are not yet aligned with the requirements of clinical use.<n>We conducted a scoping review with systematic evidence mapping across five electronic sources, screened 1,649 exportable records, and provisionally included 557 unique studies.<n>The included studies describe single agents that use external tools, supported by retrieval and external knowledge, multimodal agents, and multi-agent systems applied to medical question answering, image interpretation, electronic health record analysis, drug safety, and clinical trial prediction.
• Score: 16.678400738326236
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models and multimodal foundation models are enabling medical artificial intelligence (AI) systems to move beyond isolated prediction and undertake multistep clinical tasks that require planning, tool use, memory, iterative correction, and coordination among specialized agents. However, the scope of agentic AI in medicine remains unsettled, and current evaluation practices are not yet aligned with the requirements of clinical use. We conducted a scoping review with systematic evidence mapping across five electronic sources, screened 1,649 exportable records, and provisionally included 557 unique studies that met predefined criteria for goal-directed task execution, tool use, interaction with external resources, feedback-based refinement, or multi-agent collaboration. The included studies describe single agents that use external tools, workflows supported by retrieval and external knowledge, multimodal agents, and multi-agent systems applied to medical question answering, image interpretation, electronic health record analysis, drug safety, and clinical trial prediction. The evidence base remains dominated by public benchmarks, simulated settings, retrospective datasets, and small-scale expert evaluation. Process reliability, evidence traceability, uncertainty, safety, workflow impact, and external validity are evaluated less consistently. Clinical translation will depend on clearer definitions, reproducible evaluation, auditable oversight, interoperable system design, and prospective validation in real-world clinical workflows.

Related papers

• Evaluating Scaffolding-Oriented Multi-Agent Large Language Model System for Clinical Interview Training [10.948633957490221]
  We developed a scaffolding-oriented multi-agent Large Language Model (LLM) AI Standardized Patient (AI-SP) training platform.<n>The system includes a patient agent for simulated dialog, a tutor agent providing Socratic prompts without disclosing diagnostic information, and a turn-level evaluator agent that monitors clinical progress.<n>The most substantial and consistent improvements were observed in communication, the expression of empathy, and specific history-taking behaviors.
  arXiv  Detail & Related papers  (2026-09-10T00:56:03Z)
• Volumetric Radiology AI in the Era of Multimodal Large Language Models [62.95443321443533]
  Volumetric radiology presents a fundamental representational mismatch.<n>Current large language models (MLLMs) are conditioned on selected two-dimensional (2D) images, compressed visual representations, or report-derived text.<n>Reliable radiology AI requires representations that preserve task-relevant 3D information and systems that can access, verify, and integrate this information.
  arXiv  Detail & Related papers  (2026-08-20T20:23:41Z)
• Autonomous Information Seeking: A Roadmap for Agentic Recommender Systems [64.88044474617854]
  The rapid integration of large language model-based agents into recommender systems has driven a shift from static, ranking-based pipelines toward autonomous and interactive systems.<n>This survey introduces a unified taxonomy grounded in the level of autonomy and three core paradigms of agentic recommender systems.
  arXiv  Detail & Related papers  (2026-07-05T17:55:05Z)
• MedCTA: A Benchmark for Clinical Tool Agents [47.47354499871572]
  We introduce MedCTA, a benchmark for evaluating medical tool agents on clinician-validated, step-implicit tasks.<n>MedCTA comprises 107 real-world clinical tasks with clinician-verified executable trajectories over 5 deployed tools.<n>We benchmark 18 open- and closed-source multimodal models and find that even frontier systems remain brittle in multi-step clinical tool use.
  arXiv  Detail & Related papers  (2026-06-10T06:26:52Z)
• Evaluating Large Language Models in Dynamic Clinical Decision-Making with Standardized Patient Cases [71.12461204050985]
  MedSP1000 is an SP-derived interactive benchmark for clinical-agent evaluation.<n>It converts peer-reviewed SP teaching cases into executable scenarios with defined SP case scripts, clinical environment contexts, and human-validated structured rubrics.<n>Applying MedSP1000 to a range of general-purpose and medically specialized LLMs, we find that performance on static benchmarks does not reliably translate to such educational scenarios.
  arXiv  Detail & Related papers  (2026-06-03T17:17:16Z)
• Doctorina MedBench: End-to-End Evaluation of Agent-Based Medical AI [0.0]
  Doctorina MedBench is a comprehensive evaluation framework for agent-based medical AI based on the simulation of realistic physician-patient interactions.<n>The framework supports safety-oriented trap cases, category-based random sampling of clinical scenarios, and full regression testing.<n>The universality of the evaluation metrics allows the framework to be used not only to assess medical AI systems, but also to evaluate physicians and support the development of clinical reasoning skills.
  arXiv  Detail & Related papers  (2026-03-26T18:38:25Z)
• AgentsEval: Clinically Faithful Evaluation of Medical Imaging Reports via Multi-Agent Reasoning [73.50200033931148]
  We introduce AgentsEval, a multi-agent stream reasoning framework that emulates the collaborative diagnostic workflow of radiologists.<n>By dividing the evaluation process into interpretable steps including criteria definition, evidence extraction, alignment, and consistency scoring, AgentsEval provides explicit reasoning traces and structured clinical feedback.<n> Experimental results demonstrate that AgentsEval delivers clinically aligned, semantically faithful, and interpretable evaluations that remain robust under paraphrastic, semantic, and stylistic perturbations.
  arXiv  Detail & Related papers  (2026-01-23T11:59:13Z)
• MedDialogRubrics: A Comprehensive Benchmark and Evaluation Framework for Multi-turn Medical Consultations in Large Language Models [15.91764739198419]
  We present MedDialogRubrics, a novel benchmark comprising 5,200 synthetically constructed patient cases and over 60,000 fine-grained evaluation rubrics.<n>Our framework employs a multi-agent system to synthesize realistic patient records and chief complaints without accessing real-world electronic health records.
  arXiv  Detail & Related papers  (2026-01-06T13:56:33Z)
• Simulating Viva Voce Examinations to Evaluate Clinical Reasoning in Large Language Models [51.91760712805404]
  We introduce VivaBench, a benchmark for evaluating sequential clinical reasoning in large language models (LLMs)<n>Our dataset consists of 1762 physician-curated clinical vignettes structured as interactive scenarios that simulate a (oral) examination in medical training.<n>Our analysis identified several failure modes that mirror common cognitive errors in clinical practice.
  arXiv  Detail & Related papers  (2025-10-11T16:24:35Z)
• Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications [59.721265428780946]
  Large Language Models (LLMs) in medicine have enabled impressive capabilities, yet a critical gap remains in their ability to perform systematic, transparent, and verifiable reasoning.<n>This paper provides the first systematic review of this emerging field.<n>We propose a taxonomy of reasoning enhancement techniques, categorized into training-time strategies and test-time mechanisms.
  arXiv  Detail & Related papers  (2025-08-01T14:41:31Z)
• MedAgent-Pro: Towards Evidence-based Multi-modal Medical Diagnosis via Reasoning Agentic Workflow [14.478357882578234]
  In modern medicine, clinical diagnosis relies on the comprehensive analysis of primarily textual and visual data.<n>Recent advances in large Vision-Language Models (VLMs) and agent-based methods hold great potential for medical diagnosis.<n>We propose MedAgent-Pro, a new agentic reasoning paradigm that follows the diagnosis principle in modern medicine.
  arXiv  Detail & Related papers  (2025-03-21T14:04:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.