AI-based Clinical Decision Support for Primary Care: A Real-World Study

• URL: http://arxiv.org/abs/2507.16947v1
• Date: Tue, 22 Jul 2025 18:37:33 GMT
• Title: AI-based Clinical Decision Support for Primary Care: A Real-World Study
• Authors: Robert Korom, Sarah Kiptinness, Najib Adan, Kassim Said, Catherine Ithuli, Oliver Rotich, Boniface Kimani, Irene King'ori, Stellah Kamau, Elizabeth Atemba, Muna Aden, Preston Bowman, Michael Sharman, Rebecca Soskin Hicks, Rebecca Distler, Johannes Heidecke, Rahul K. Arora, Karan Singhal,
• Abstract summary: We evaluate the impact of large language model-based clinical decision support in live care.<n>We studied AI Consult, a tool that serves as a safety net for clinicians by identifying potential documentation and clinical decision-making errors.
• Score: 1.2764851761863103
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We evaluate the impact of large language model-based clinical decision support in live care. In partnership with Penda Health, a network of primary care clinics in Nairobi, Kenya, we studied AI Consult, a tool that serves as a safety net for clinicians by identifying potential documentation and clinical decision-making errors. AI Consult integrates into clinician workflows, activating only when needed and preserving clinician autonomy. We conducted a quality improvement study, comparing outcomes for 39,849 patient visits performed by clinicians with or without access to AI Consult across 15 clinics. Visits were rated by independent physicians to identify clinical errors. Clinicians with access to AI Consult made relatively fewer errors: 16% fewer diagnostic errors and 13% fewer treatment errors. In absolute terms, the introduction of AI Consult would avert diagnostic errors in 22,000 visits and treatment errors in 29,000 visits annually at Penda alone. In a survey of clinicians with AI Consult, all clinicians said that AI Consult improved the quality of care they delivered, with 75% saying the effect was "substantial". These results required a clinical workflow-aligned AI Consult implementation and active deployment to encourage clinician uptake. We hope this study demonstrates the potential for LLM-based clinical decision support tools to reduce errors in real-world settings and provides a practical framework for advancing responsible adoption.

Related papers

• Toward the Autonomous AI Doctor: Quantitative Benchmarking of an Autonomous Agentic AI Versus Board-Certified Clinicians in a Real World Setting [0.0]
  Globally we face a projected shortage of 11 million healthcare practitioners by 2030.<n>No end-to-end autonomous large language model (LLM)-based AI system has been rigorously evaluated in real-world clinical practice.
  arXiv  Detail & Related papers  (2025-06-27T19:04:44Z)
• Designing AI Tools for Clinical Care Teams to Support Serious Illness Conversations with Older Adults in the Emergency Department [53.52248484568777]
  The work contributes empirical understanding of ED-based serious illness conversations and provides design considerations for AI in high-stakes clinical environments.<n>We conducted interviews with two domain experts and nine ED clinical care team members.<n>We characterized a four-phase serious illness conversation workflow (identification, preparation, conduction, documentation) and identified key needs and challenges at each stage.<n>We present design guidelines for AI tools supporting SIC that fit within existing clinical practices.
  arXiv  Detail & Related papers  (2025-05-30T21:15:57Z)
• Over-Relying on Reliance: Towards Realistic Evaluations of AI-Based Clinical Decision Support [12.247046469627554]
  We advocate for moving beyond evaluation metrics like Trust, Reliance, Acceptance, and Performance on the AI's task.<n>We call on the community to prioritize ecologically valid, domain-appropriate study setups that measure the emergent forms of value that AI can bring to healthcare professionals.
  arXiv  Detail & Related papers  (2025-04-10T03:28:56Z)
• Clinicians' Voice: Fundamental Considerations for XAI in Healthcare [0.0]
  We conducted semi-structured interviews with clinicians to discuss their thoughts, hopes, and concerns.<n> Clinicians from our sample generally think positively about developing AI-based tools for clinical practice.<n>They have concerns about how these will fit into their workflow and how it will impact clinician-patient relations.
  arXiv  Detail & Related papers  (2024-11-07T16:47:06Z)
• TrialBench: Multi-Modal Artificial Intelligence-Ready Clinical Trial Datasets [54.98321887435557]
  This paper presents a suite of 23 meticulously curated AI-ready datasets covering multi-modal input features and 8 crucial prediction challenges in clinical trial design.<n>We provide basic validation methods for each task to ensure the datasets' usability and reliability.<n>We anticipate that the availability of such open-access datasets will catalyze the development of advanced AI approaches for clinical trial design.
  arXiv  Detail & Related papers  (2024-06-30T09:13:10Z)
• Large Language Models in the Clinic: A Comprehensive Benchmark [63.21278434331952]
  We build a benchmark ClinicBench to better understand large language models (LLMs) in the clinic. We first collect eleven existing datasets covering diverse clinical language generation, understanding, and reasoning tasks. We then construct six novel datasets and clinical tasks that are complex but common in real-world practice. We conduct an extensive evaluation of twenty-two LLMs under both zero-shot and few-shot settings.
  arXiv  Detail & Related papers  (2024-04-25T15:51:06Z)
• AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator [69.51568871044454]
  We introduce textbfAI Hospital, a framework simulating dynamic medical interactions between emphDoctor as player and NPCs. This setup allows for realistic assessments of LLMs in clinical scenarios. We develop the Multi-View Medical Evaluation benchmark, utilizing high-quality Chinese medical records and NPCs.
  arXiv  Detail & Related papers  (2024-02-15T06:46:48Z)
• Towards Conversational Diagnostic AI [32.84876349808714]
  We introduce AMIE (Articulate Medical Intelligence Explorer), a Large Language Model (LLM) based AI system optimized for diagnostic dialogue. AMIE uses a self-play based simulated environment with automated feedback mechanisms for scaling learning across diverse disease conditions. AMIE demonstrated greater diagnostic accuracy and superior performance on 28 of 32 axes according to specialist physicians and 24 of 26 axes according to patient actors.
  arXiv  Detail & Related papers  (2024-01-11T04:25:06Z)
• Towards Accurate Differential Diagnosis with Large Language Models [37.48155380562073]
  Interactive interfaces powered by Large Language Models (LLMs) present new opportunities to both assist and automate aspects of differential diagnosis. 20 clinicians evaluated 302 challenging, real-world medical cases sourced from the New England Journal of Medicine. Our study suggests that our LLM has potential to improve clinicians' diagnostic reasoning and accuracy in challenging cases.
  arXiv  Detail & Related papers  (2023-11-30T19:55:51Z)
• Advancing COVID-19 Diagnosis with Privacy-Preserving Collaboration in Artificial Intelligence [79.038671794961]
  We launch the Unified CT-COVID AI Diagnostic Initiative (UCADI), where the AI model can be distributedly trained and independently executed at each host institution. Our study is based on 9,573 chest computed tomography scans (CTs) from 3,336 patients collected from 23 hospitals located in China and the UK.
  arXiv  Detail & Related papers  (2021-11-18T00:43:41Z)
• COVID-Net Clinical ICU: Enhanced Prediction of ICU Admission for COVID-19 Patients via Explainability and Trust Quantification [71.80459780697956]
  We introduce COVID-Net Clinical ICU, a neural network for ICU admission prediction based on patient clinical data. The proposed COVID-Net Clinical ICU was built using a clinical dataset from Hospital Sirio-Libanes comprising of 1,925 COVID-19 patients. We conducted system-level insight discovery using a quantitative explainability strategy to study the decision-making impact of different clinical features.
  arXiv  Detail & Related papers  (2021-09-14T14:16:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.