AI-based Clinical Decision Support for Primary Care: A Real-World Study

• URL: http://arxiv.org/abs/2507.16947v1
• Date: Tue, 22 Jul 2025 18:37:33 GMT
• Title: AI-based Clinical Decision Support for Primary Care: A Real-World Study
• Abstract summary: We evaluate the impact of large language model-based clinical decision support in live care.<n>We studied AI Consult, a tool that serves as a safety net for clinicians by identifying potential documentation and clinical decision-making errors.
• Score: 1.2764851761863103
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We evaluate the impact of large language model-based clinical decision support in live care. In partnership with Penda Health, a network of primary care clinics in Nairobi, Kenya, we studied AI Consult, a tool that serves as a safety net for clinicians by identifying potential documentation and clinical decision-making errors. AI Consult integrates into clinician workflows, activating only when needed and preserving clinician autonomy. We conducted a quality improvement study, comparing outcomes for 39,849 patient visits performed by clinicians with or without access to AI Consult across 15 clinics. Visits were rated by independent physicians to identify clinical errors. Clinicians with access to AI Consult made relatively fewer errors: 16% fewer diagnostic errors and 13% fewer treatment errors. In absolute terms, the introduction of AI Consult would avert diagnostic errors in 22,000 visits and treatment errors in 29,000 visits annually at Penda alone. In a survey of clinicians with AI Consult, all clinicians said that AI Consult improved the quality of care they delivered, with 75% saying the effect was "substantial". These results required a clinical workflow-aligned AI Consult implementation and active deployment to encourage clinician uptake. We hope this study demonstrates the potential for LLM-based clinical decision support tools to reduce errors in real-world settings and provides a practical framework for advancing responsible adoption.

Related papers

• Assessment of Non-Institutional AI Tool Usage Among Clinicians [0.0]
  Generative artificial intelligence (AI) tools are increasingly accessible and have the potential to improve efficiency across clinical.<n>However, clinicians may also use non-institutional AI tools that are not provided, managed, or governed by their healthcare institutions.<n>We conducted a survey of clinicians recruited from the University of Arizona College of Medicine-Tucson and Banner University Medical Center-Tucson.
  arXiv  Detail & Related papers  (2026-09-08T22:47:34Z)
• Towards AI-Assisted Clinical Trial Matching: Practical Considerations, Multicenter Evaluation, and Real-World Deployment [17.27078524771392]
  We present TrialGPT 2.0, an AI-assisted clinical trial recommendation system designed for real-world deployment.<n>Rather than asking only whether a patient may qualify, TrialGPT 2.0 assesses which trials warrant further consideration.<n>TrialGPT 2.0 retrieved at least one clinician-recommended trial in its top 10 recommendations for approximately 91% of cases.
  arXiv  Detail & Related papers  (2026-09-01T13:11:02Z)
• ResidencyRL: Reinforcement Learning in Simulated Clinical Environments [53.76751756107077]
  We present ResidencyRL, a reinforcement learning (RL) method for training clinical artificial intelligence (AI) agents through simulated multi-turn clinical encounters.<n>On heldout evaluations, the ResidencyRL agent improves diagnostic accuracy by 7.0% under adversarial conditions.<n>Blinded expert clinicians validated these gains, preferring the trained agent in 87.6% of side-by-side comparisons.
  arXiv  Detail & Related papers  (2026-08-07T17:04:41Z)
• Augmenting Clinical Decision-Making with an Interactive and Interpretable AI Copilot: A Real-World User Study with Clinicians in Nephrology and Obstetrics [36.981753143345664]
  We present AICare, an interactive and interpretable AI copilot for collaborative clinical decision-making.<n>By analyzing longitudinal electronic health records, AICare grounds dynamic risk predictions in scrutable visualizations.
  arXiv  Detail & Related papers  (2026-01-31T13:41:32Z)
• Simulating Viva Voce Examinations to Evaluate Clinical Reasoning in Large Language Models [51.91760712805404]
  We introduce VivaBench, a benchmark for evaluating sequential clinical reasoning in large language models (LLMs)<n>Our dataset consists of 1762 physician-curated clinical vignettes structured as interactive scenarios that simulate a (oral) examination in medical training.<n>Our analysis identified several failure modes that mirror common cognitive errors in clinical practice.
  arXiv  Detail & Related papers  (2025-10-11T16:24:35Z)
• Reverse Physician-AI Relationship: Full-process Clinical Diagnosis Driven by a Large Language Model [71.40113970879219]
  We propose a paradigm shift that reverses the relationship between physicians and AI.<n>We present DxDirector-7B, an LLM endowed with advanced deep thinking capabilities, enabling it to drive the full-process diagnosis with minimal physician involvement.<n>In evaluations across rare, complex, and real-world cases under full-process diagnosis setting, DxDirector-7B not only achieves significant superior diagnostic accuracy but also substantially reduces physician workload.
  arXiv  Detail & Related papers  (2025-08-14T09:51:20Z)
• Toward the Autonomous AI Doctor: Quantitative Benchmarking of an Autonomous Agentic AI Versus Board-Certified Clinicians in a Real World Setting [0.0]
  Globally we face a projected shortage of 11 million healthcare practitioners by 2030.<n>No end-to-end autonomous large language model (LLM)-based AI system has been rigorously evaluated in real-world clinical practice.
  arXiv  Detail & Related papers  (2025-06-27T19:04:44Z)
• Designing AI Tools for Clinical Care Teams to Support Serious Illness Conversations with Older Adults in the Emergency Department [53.52248484568777]
  The work contributes empirical understanding of ED-based serious illness conversations and provides design considerations for AI in high-stakes clinical environments.<n>We conducted interviews with two domain experts and nine ED clinical care team members.<n>We characterized a four-phase serious illness conversation workflow (identification, preparation, conduction, documentation) and identified key needs and challenges at each stage.<n>We present design guidelines for AI tools supporting SIC that fit within existing clinical practices.
  arXiv  Detail & Related papers  (2025-05-30T21:15:57Z)
• Over-Relying on Reliance: Towards Realistic Evaluations of AI-Based Clinical Decision Support [12.247046469627554]
  We advocate for moving beyond evaluation metrics like Trust, Reliance, Acceptance, and Performance on the AI's task.<n>We call on the community to prioritize ecologically valid, domain-appropriate study setups that measure the emergent forms of value that AI can bring to healthcare professionals.
  arXiv  Detail & Related papers  (2025-04-10T03:28:56Z)
• Clinicians' Voice: Fundamental Considerations for XAI in Healthcare [0.0]
  We conducted semi-structured interviews with clinicians to discuss their thoughts, hopes, and concerns.<n> Clinicians from our sample generally think positively about developing AI-based tools for clinical practice.<n>They have concerns about how these will fit into their workflow and how it will impact clinician-patient relations.
  arXiv  Detail & Related papers  (2024-11-07T16:47:06Z)
• TrialBench: Multi-Modal Artificial Intelligence-Ready Clinical Trial Datasets [54.98321887435557]
  This paper presents a suite of 23 meticulously curated AI-ready datasets covering multi-modal input features and 8 crucial prediction challenges in clinical trial design.<n>We provide basic validation methods for each task to ensure the datasets' usability and reliability.<n>We anticipate that the availability of such open-access datasets will catalyze the development of advanced AI approaches for clinical trial design.
  arXiv  Detail & Related papers  (2024-06-30T09:13:10Z)
• Large Language Models in the Clinic: A Comprehensive Benchmark [63.21278434331952]
  We build a benchmark ClinicBench to better understand large language models (LLMs) in the clinic. We first collect eleven existing datasets covering diverse clinical language generation, understanding, and reasoning tasks. We then construct six novel datasets and clinical tasks that are complex but common in real-world practice. We conduct an extensive evaluation of twenty-two LLMs under both zero-shot and few-shot settings.
  arXiv  Detail & Related papers  (2024-04-25T15:51:06Z)
• AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator [69.51568871044454]
  We introduce textbfAI Hospital, a framework simulating dynamic medical interactions between emphDoctor as player and NPCs. This setup allows for realistic assessments of LLMs in clinical scenarios. We develop the Multi-View Medical Evaluation benchmark, utilizing high-quality Chinese medical records and NPCs.
  arXiv  Detail & Related papers  (2024-02-15T06:46:48Z)
• Towards Conversational Diagnostic AI [32.84876349808714]
  We introduce AMIE (Articulate Medical Intelligence Explorer), a Large Language Model (LLM) based AI system optimized for diagnostic dialogue. AMIE uses a self-play based simulated environment with automated feedback mechanisms for scaling learning across diverse disease conditions. AMIE demonstrated greater diagnostic accuracy and superior performance on 28 of 32 axes according to specialist physicians and 24 of 26 axes according to patient actors.
  arXiv  Detail & Related papers  (2024-01-11T04:25:06Z)
• Towards Accurate Differential Diagnosis with Large Language Models [37.48155380562073]
  Interactive interfaces powered by Large Language Models (LLMs) present new opportunities to both assist and automate aspects of differential diagnosis. 20 clinicians evaluated 302 challenging, real-world medical cases sourced from the New England Journal of Medicine. Our study suggests that our LLM has potential to improve clinicians' diagnostic reasoning and accuracy in challenging cases.
  arXiv  Detail & Related papers  (2023-11-30T19:55:51Z)
• Towards the Use of Saliency Maps for Explaining Low-Quality Electrocardiograms to End Users [51.644376281196394]
  When using medical images for diagnosis, it is important that the images are of high quality.<n>In telemedicine, a common problem is that the quality issue is only flagged once the patient has left the clinic, meaning they must return in order to have the exam redone.<n>This paper reports on the development of an AI system for flagging and explaining low-quality medical images in real-time.
  arXiv  Detail & Related papers  (2022-07-06T14:53:26Z)
• Advancing COVID-19 Diagnosis with Privacy-Preserving Collaboration in Artificial Intelligence [79.038671794961]
  We launch the Unified CT-COVID AI Diagnostic Initiative (UCADI), where the AI model can be distributedly trained and independently executed at each host institution. Our study is based on 9,573 chest computed tomography scans (CTs) from 3,336 patients collected from 23 hospitals located in China and the UK.
  arXiv  Detail & Related papers  (2021-11-18T00:43:41Z)
• COVID-Net Clinical ICU: Enhanced Prediction of ICU Admission for COVID-19 Patients via Explainability and Trust Quantification [71.80459780697956]
  We introduce COVID-Net Clinical ICU, a neural network for ICU admission prediction based on patient clinical data. The proposed COVID-Net Clinical ICU was built using a clinical dataset from Hospital Sirio-Libanes comprising of 1,925 COVID-19 patients. We conducted system-level insight discovery using a quantitative explainability strategy to study the decision-making impact of different clinical features.
  arXiv  Detail & Related papers  (2021-09-14T14:16:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.