Enhancing Large Language Models for Clinical Decision Support by Incorporating Clinical Practice Guidelines

• URL: http://arxiv.org/abs/2401.11120v2
• Date: Tue, 23 Jan 2024 19:43:06 GMT
• Title: Enhancing Large Language Models for Clinical Decision Support by Incorporating Clinical Practice Guidelines
• Abstract summary: Large Language Models (LLMs) can significantly improve Clinical Decision Support (CDS) We develop three distinct methods for incorporating Clinical Practice Guidelines (CPGs) into LLMs. We focus on CDS for COVID-19 outpatient treatment as the case study.
• Score: 4.482833270578723
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Background Large Language Models (LLMs), enhanced with Clinical Practice Guidelines (CPGs), can significantly improve Clinical Decision Support (CDS). However, methods for incorporating CPGs into LLMs are not well studied. Methods We develop three distinct methods for incorporating CPGs into LLMs: Binary Decision Tree (BDT), Program-Aided Graph Construction (PAGC), and Chain-of-Thought-Few-Shot Prompting (CoT-FSP). To evaluate the effectiveness of the proposed methods, we create a set of synthetic patient descriptions and conduct both automatic and human evaluation of the responses generated by four LLMs: GPT-4, GPT-3.5 Turbo, LLaMA, and PaLM 2. Zero-Shot Prompting (ZSP) was used as the baseline method. We focus on CDS for COVID-19 outpatient treatment as the case study. Results All four LLMs exhibit improved performance when enhanced with CPGs compared to the baseline ZSP. BDT outperformed both CoT-FSP and PAGC in automatic evaluation. All of the proposed methods demonstrated high performance in human evaluation. Conclusion LLMs enhanced with CPGs demonstrate superior performance, as compared to plain LLMs with ZSP, in providing accurate recommendations for COVID-19 outpatient treatment, which also highlights the potential for broader applications beyond the case study.

Related papers

• An Iterative LLM Framework for SIBT utilizing RAG-based Adaptive Weight Optimization [11.168299220031662]
  This study proposes an adaptive weight optimization framework for SIBT planning, driven by large language models (LLMs)<n>A clinical knowledge base, constructed and queried via retrieval-augmented generation (RAG), enhances the model's domain-specific reasoning.<n>The proposed method was validated on 23 patient cases, showing that the LLM-assisted approach produces plans that are comparable to or exceeding clinically approved and fixed-weight plans.
  arXiv  Detail & Related papers  (2025-09-10T08:54:16Z)
• The Thinking Therapist: Training Large Language Models to Deliver Acceptance and Commitment Therapy using Supervised Fine-Tuning and Odds Ratio Policy Optimization [0.0]
  Acceptance and Commitment Therapy (ACT) is a cognitive behavioral therapy with emerging evidence of efficacy in several psychiatric conditions.<n>This study investigates the impact of post-training methodology and explicit reasoning on the ability of a small open-weight large language model (LLM) to deliver ACT.
  arXiv  Detail & Related papers  (2025-09-08T02:30:12Z)
• When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness Methods for LLMs [55.20230501807337]
  We present the first systematic evaluation of 5 methods for improving prompt robustness within a unified experimental framework.<n>We benchmark these techniques on 8 models from Llama, Qwen and Gemma families across 52 tasks from Natural Instructions dataset.
  arXiv  Detail & Related papers  (2025-08-15T10:32:50Z)
• Iterative Learning of Computable Phenotypes for Treatment Resistant Hypertension using Large Language Models [1.4952056744888915]
  Large language models (LLMs) have demonstrated remarkable capabilities for medical question answering and programming.<n>We investigate whether LLMs can generate accurate and concise CPs for six clinical phenotypes of varying complexity.<n>Our results show that LLMs, coupled with iterative learning, can generate interpretable and reasonably accurate programs.
  arXiv  Detail & Related papers  (2025-08-07T17:15:17Z)
• ChestX-Reasoner: Advancing Radiology Foundation Models with Reasoning through Step-by-Step Verification [57.22053411719822]
  ChestX-Reasoner is a radiology diagnosis MLLM designed to leverage process supervision mined directly from clinical reports. Our two-stage training framework combines supervised fine-tuning and reinforcement learning guided by process rewards to better align model reasoning with clinical standards.
  arXiv  Detail & Related papers  (2025-04-29T16:48:23Z)
• LlaMADRS: Prompting Large Language Models for Interview-Based Depression Assessment [75.44934940580112]
  This study introduces LlaMADRS, a novel framework leveraging open-source Large Language Models (LLMs) to automate depression severity assessment. We employ a zero-shot prompting strategy with carefully designed cues to guide the model in interpreting and scoring transcribed clinical interviews. Our approach, tested on 236 real-world interviews, demonstrates strong correlations with clinician assessments.
  arXiv  Detail & Related papers  (2025-01-07T08:49:04Z)
• A Comparative Study of Recent Large Language Models on Generating Hospital Discharge Summaries for Lung Cancer Patients [19.777109737517996]
  This research aims to explore how large language models (LLMs) can alleviate the burden of manual summarization. This study evaluates the performance of multiple LLMs, including GPT-3.5, GPT-4, GPT-4o, and LLaMA 3 8b, in generating discharge summaries.
  arXiv  Detail & Related papers  (2024-11-06T10:02:50Z)
• SeRTS: Self-Rewarding Tree Search for Biomedical Retrieval-Augmented Generation [50.26966969163348]
  Large Language Models (LLMs) have shown great potential in the biomedical domain with the advancement of retrieval-augmented generation (RAG) Existing retrieval-augmented approaches face challenges in addressing diverse queries and documents, particularly for medical knowledge queries. We propose Self-Rewarding Tree Search (SeRTS) based on Monte Carlo Tree Search (MCTS) and a self-rewarding paradigm.
  arXiv  Detail & Related papers  (2024-06-17T06:48:31Z)
• A Dataset and Benchmark for Hospital Course Summarization with Adapted Large Language Models [4.091402760759184]
  Large language models (LLMs) depict remarkable capabilities in automating real-world tasks, but their capabilities for healthcare applications have not been shown. We introduce a novel pre-processed dataset, the MIMIC-IV-BHC, encapsulating clinical note and brief hospital course (BHC) pairs to adapt LLMs for BHC. Using clinical notes as input, we apply prompting-based (using in-context learning) and fine-tuning-based adaptation strategies to three open-source LLMs and two proprietary LLMs.
  arXiv  Detail & Related papers  (2024-03-08T23:17:55Z)
• Evaluation of General Large Language Models in Contextually Assessing Semantic Concepts Extracted from Adult Critical Care Electronic Health Record Notes [17.648021186810663]
  The purpose of this study was to evaluate the performance of Large Language Models (LLMs) in understanding and processing real-world clinical notes. The GPT family models have demonstrated considerable efficiency, evidenced by their cost-effectiveness and time-saving capabilities.
  arXiv  Detail & Related papers  (2024-01-24T16:52:37Z)
• Distilling Large Language Models for Matching Patients to Clinical Trials [3.4068841624198942]
  The recent success of large language models (LLMs) has paved the way for their adoption in the high-stakes domain of healthcare. This study presents the first systematic examination of the efficacy of both proprietary (GPT-3.5, and GPT-4) and open-source LLMs (LLAMA 7B,13B, and 70B) for the task of patient-trial matching. Our findings reveal that open-source LLMs, when fine-tuned on this limited and synthetic dataset, demonstrate performance parity with their proprietary counterparts.
  arXiv  Detail & Related papers  (2023-12-15T17:11:07Z)
• Using Natural Language Explanations to Improve Robustness of In-context Learning [35.18010811754959]
  Large language models (LLMs) can excel in many tasks via in-context learning (ICL) We investigate whether augmenting ICL with natural language explanations (NLEs) improves the robustness of LLMs on adversarial datasets.
  arXiv  Detail & Related papers  (2023-11-13T18:49:13Z)
• Automated Assessment of Critical View of Safety in Laparoscopic Cholecystectomy [51.240181118593114]
  Cholecystectomy (gallbladder removal) is one of the most common procedures in the US, with more than 1.2M procedures annually. LC is associated with an increase in bile duct injuries (BDIs), resulting in significant morbidity and mortality. In this paper, we develop deep-learning techniques to automate the assessment of critical view of safety (CVS) in LCs.
  arXiv  Detail & Related papers  (2023-09-13T22:01:36Z)
• LLMRec: Benchmarking Large Language Models on Recommendation Task [54.48899723591296]
  The application of Large Language Models (LLMs) in the recommendation domain has not been thoroughly investigated. We benchmark several popular off-the-shelf LLMs on five recommendation tasks, including rating prediction, sequential recommendation, direct recommendation, explanation generation, and review summarization. The benchmark results indicate that LLMs displayed only moderate proficiency in accuracy-based tasks such as sequential and direct recommendation.
  arXiv  Detail & Related papers  (2023-08-23T16:32:54Z)
• Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting [65.00288634420812]
  Pairwise Ranking Prompting (PRP) is a technique to significantly reduce the burden on Large Language Models (LLMs) Our results are the first in the literature to achieve state-of-the-art ranking performance on standard benchmarks using moderate-sized open-sourced LLMs.
  arXiv  Detail & Related papers  (2023-06-30T11:32:25Z)
• Large Language Models for Healthcare Data Augmentation: An Example on Patient-Trial Matching [49.78442796596806]
  We propose an innovative privacy-aware data augmentation approach for patient-trial matching (LLM-PTM) Our experiments demonstrate a 7.32% average improvement in performance using the proposed LLM-PTM method, and the generalizability to new data is improved by 12.12%.
  arXiv  Detail & Related papers  (2023-03-24T03:14:00Z)
• Performance of Dual-Augmented Lagrangian Method and Common Spatial Patterns applied in classification of Motor-Imagery BCI [68.8204255655161]
  Motor-imagery based brain-computer interfaces (MI-BCI) have the potential to become ground-breaking technologies for neurorehabilitation. Due to the noisy nature of the used EEG signal, reliable BCI systems require specialized procedures for features optimization and extraction.
  arXiv  Detail & Related papers  (2020-10-13T20:50:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.