Controlled LLM-based Reasoning for Clinical Trial Retrieval

• URL: http://arxiv.org/abs/2409.18998v1
• Date: Thu, 19 Sep 2024 09:42:33 GMT
• Title: Controlled LLM-based Reasoning for Clinical Trial Retrieval
• Authors: Mael Jullien, Alex Bogatu, Harriet Unsworth, Andre Freitas,
• Abstract summary: We propose a scalable method that extends the capabilities of LLMs in the direction of systematizing the reasoning over sets of medical eligibility criteria. The proposed method is evaluated on TREC 2022 Clinical Trials, achieving results superior to the state-of-the-art: NDCG@10 of 0.693 and Precision@10 of 0.73.
• Score: 0.4199844472131922
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Matching patients to clinical trials demands a systematic and reasoned interpretation of documents which require significant expert-level background knowledge, over a complex set of well-defined eligibility criteria. Moreover, this interpretation process needs to operate at scale, over vast knowledge bases of trials. In this paper, we propose a scalable method that extends the capabilities of LLMs in the direction of systematizing the reasoning over sets of medical eligibility criteria, evaluating it in the context of real-world cases. The proposed method overlays a Set-guided reasoning method for LLMs. The proposed framework is evaluated on TREC 2022 Clinical Trials, achieving results superior to the state-of-the-art: NDCG@10 of 0.693 and Precision@10 of 0.73.

Related papers

• Comprehensive and Practical Evaluation of Retrieval-Augmented Generation Systems for Medical Question Answering [70.44269982045415]
  Retrieval-augmented generation (RAG) has emerged as a promising approach to enhance the performance of large language models (LLMs) We introduce Medical Retrieval-Augmented Generation Benchmark (MedRGB) that provides various supplementary elements to four medical QA datasets. Our experimental results reveals current models' limited ability to handle noise and misinformation in the retrieved documents.
  arXiv  Detail & Related papers  (2024-11-14T06:19:18Z)
• Demystifying Large Language Models for Medicine: A Primer [50.83806796466396]
  Large language models (LLMs) represent a transformative class of AI tools capable of revolutionizing various aspects of healthcare. This tutorial aims to equip healthcare professionals with the tools necessary to effectively integrate LLMs into clinical practice.
  arXiv  Detail & Related papers  (2024-10-24T15:41:56Z)
• Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrieval [61.70489848327436]
  KARE is a novel framework that integrates knowledge graph (KG) community-level retrieval with large language models (LLMs) reasoning. Extensive experiments demonstrate that KARE outperforms leading models by up to 10.8-15.0% on MIMIC-III and 12.6-12.7% on MIMIC-IV for mortality and readmission predictions.
  arXiv  Detail & Related papers  (2024-10-06T18:46:28Z)
• CliMedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models in Clinical Scenarios [50.032101237019205]
  CliMedBench is a comprehensive benchmark with 14 expert-guided core clinical scenarios. The reliability of this benchmark has been confirmed in several ways.
  arXiv  Detail & Related papers  (2024-10-04T15:15:36Z)
• SemioLLM: Assessing Large Language Models for Semiological Analysis in Epilepsy Research [45.2233252981348]
  Large Language Models have shown promising results in their ability to encode general medical knowledge. We test the ability of state-of-the-art LLMs to leverage their internal knowledge and reasoning for epilepsy diagnosis.
  arXiv  Detail & Related papers  (2024-07-03T11:02:12Z)
• Towards Automatic Evaluation for LLMs' Clinical Capabilities: Metric, Data, and Algorithm [15.627870862369784]
  Large language models (LLMs) are gaining increasing interests to improve clinical efficiency for medical diagnosis. We propose an automatic evaluation paradigm tailored to assess the LLMs' capabilities in delivering clinical services.
  arXiv  Detail & Related papers  (2024-03-25T06:17:54Z)
• Zero-Shot Clinical Trial Patient Matching with LLMs [40.31971412825736]
  Large language models (LLMs) offer a promising solution to automated screening. We design an LLM-based system which, given a patient's medical history as unstructured clinical text, evaluates whether that patient meets a set of inclusion criteria. Our system achieves state-of-the-art scores on the n2c2 2018 cohort selection benchmark.
  arXiv  Detail & Related papers  (2024-02-05T00:06:08Z)
• An Automatic Evaluation Framework for Multi-turn Medical Consultations Capabilities of Large Language Models [22.409334091186995]
  Large language models (LLMs) often suffer from hallucinations, leading to overly confident but incorrect judgments. This paper introduces an automated evaluation framework that assesses the practical capabilities of LLMs as virtual doctors during multi-turn consultations.
  arXiv  Detail & Related papers  (2023-09-05T09:24:48Z)
• Scaling Clinical Trial Matching Using Large Language Models: A Case Study in Oncology [17.214664001970526]
  We conduct a systematic study on scaling clinical trial matching using large language models (LLMs) Our study is grounded in a clinical trial matching system currently in test deployment at a large U.S. health network.
  arXiv  Detail & Related papers  (2023-08-04T07:51:15Z)
• AutoTrial: Prompting Language Models for Clinical Trial Design [53.630479619856516]
  We present a method named AutoTrial to aid the design of clinical eligibility criteria using language models. Experiments on over 70K clinical trials verify that AutoTrial generates high-quality criteria texts.
  arXiv  Detail & Related papers  (2023-05-19T01:04:16Z)
• Improving Patient Pre-screening for Clinical Trials: Assisting Physicians with Large Language Models [0.0]
  Large Language Models (LLMs) have shown to perform well for clinical information extraction and clinical reasoning. This paper investigates the use of InstructGPT to assist physicians in determining eligibility for clinical trials based on a patient's summarised medical profile.
  arXiv  Detail & Related papers  (2023-04-14T21:19:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.