Clinical trial cohort selection using Large Language Models on n2c2 Challenges

• URL: http://arxiv.org/abs/2501.11114v1
• Date: Sun, 19 Jan 2025 17:07:02 GMT
• Title: Clinical trial cohort selection using Large Language Models on n2c2 Challenges
• Authors: Chi-en Amy Tai, Xavier Tannier,
• Abstract summary: Large language models (LLMs) have gained popularity for various NLP tasks due to their ability to acquire a nuanced understanding of text. Our results are promising with regard to the incorporation of LLMs for simple cohort selection tasks, but also highlight the difficulties encountered by these models as soon as fine-grained knowledge and reasoning are required.
• Score: 3.0208841164563838
• License:
• Abstract: Clinical trials are a critical process in the medical field for introducing new treatments and innovations. However, cohort selection for clinical trials is a time-consuming process that often requires manual review of patient text records for specific keywords. Though there have been studies on standardizing the information across the various platforms, Natural Language Processing (NLP) tools remain crucial for spotting eligibility criteria in textual reports. Recently, pre-trained large language models (LLMs) have gained popularity for various NLP tasks due to their ability to acquire a nuanced understanding of text. In this paper, we study the performance of large language models on clinical trial cohort selection and leverage the n2c2 challenges to benchmark their performance. Our results are promising with regard to the incorporation of LLMs for simple cohort selection tasks, but also highlight the difficulties encountered by these models as soon as fine-grained knowledge and reasoning are required.

Related papers

• The use of large language models to enhance cancer clinical trial educational materials [2.680807601066252]
  GPT4-generated trial summaries were both readable and comprehensive. Multiple-choice questions demonstrated high accuracy and agreement with crowdsourced annotators. For both resource types, hallucinations were identified that require ongoing human oversight.
  arXiv  Detail & Related papers  (2024-12-02T20:31:27Z)
• Demystifying Large Language Models for Medicine: A Primer [50.83806796466396]
  Large language models (LLMs) represent a transformative class of AI tools capable of revolutionizing various aspects of healthcare. This tutorial aims to equip healthcare professionals with the tools necessary to effectively integrate LLMs into clinical practice.
  arXiv  Detail & Related papers  (2024-10-24T15:41:56Z)
• Large Language Models in the Clinic: A Comprehensive Benchmark [63.21278434331952]
  We build a benchmark ClinicBench to better understand large language models (LLMs) in the clinic. We first collect eleven existing datasets covering diverse clinical language generation, understanding, and reasoning tasks. We then construct six novel datasets and clinical tasks that are complex but common in real-world practice. We conduct an extensive evaluation of twenty-two LLMs under both zero-shot and few-shot settings.
  arXiv  Detail & Related papers  (2024-04-25T15:51:06Z)
• Zero-Shot Clinical Trial Patient Matching with LLMs [40.31971412825736]
  Large language models (LLMs) offer a promising solution to automated screening. We design an LLM-based system which, given a patient's medical history as unstructured clinical text, evaluates whether that patient meets a set of inclusion criteria. Our system achieves state-of-the-art scores on the n2c2 2018 cohort selection benchmark.
  arXiv  Detail & Related papers  (2024-02-05T00:06:08Z)
• CohortGPT: An Enhanced GPT for Participant Recruitment in Clinical Study [17.96401880059829]
  Large Language Models (LLMs) such as ChatGPT have achieved tremendous success in various downstream tasks. We propose to use a knowledge graph as auxiliary information to guide the LLMs in making predictions. Our few-shot learning method achieves satisfactory performance compared with fine-tuning strategies.
  arXiv  Detail & Related papers  (2023-07-21T04:43:00Z)
• Self-Verification Improves Few-Shot Clinical Information Extraction [73.6905567014859]
  Large language models (LLMs) have shown the potential to accelerate clinical curation via few-shot in-context learning. They still struggle with issues regarding accuracy and interpretability, especially in mission-critical domains such as health. Here, we explore a general mitigation framework using self-verification, which leverages the LLM to provide provenance for its own extraction and check its own outputs.
  arXiv  Detail & Related papers  (2023-05-30T22:05:11Z)
• Large Language Models for Healthcare Data Augmentation: An Example on Patient-Trial Matching [49.78442796596806]
  We propose an innovative privacy-aware data augmentation approach for patient-trial matching (LLM-PTM) Our experiments demonstrate a 7.32% average improvement in performance using the proposed LLM-PTM method, and the generalizability to new data is improved by 12.12%.
  arXiv  Detail & Related papers  (2023-03-24T03:14:00Z)
• Few-Shot Learning for Clinical Natural Language Processing Using Siamese Neural Networks [3.9586758145580014]
  Clinical Natural Language Processing (NLP) has become an emerging technology in healthcare. Deep learning has achieved state-of-the-art performance in many clinical NLP tasks. Training deep learning models usually require large annotated datasets, which are normally not publicly available.
  arXiv  Detail & Related papers  (2022-08-31T15:36:27Z)
• HealthPrompt: A Zero-shot Learning Paradigm for Clinical Natural Language Processing [3.762895631262445]
  We developed a novel prompt-based clinical NLP framework called HealthPrompt. We performed an in-depth analysis of HealthPrompt on six different PLMs in a no-data setting. Our experiments prove that prompts effectively capture the context of clinical texts and perform remarkably well without any training data.
  arXiv  Detail & Related papers  (2022-03-09T21:44:28Z)
• VBridge: Connecting the Dots Between Features, Explanations, and Data for Healthcare Models [85.4333256782337]
  VBridge is a visual analytics tool that seamlessly incorporates machine learning explanations into clinicians' decision-making workflow. We identified three key challenges, including clinicians' unfamiliarity with ML features, lack of contextual information, and the need for cohort-level evidence. We demonstrated the effectiveness of VBridge through two case studies and expert interviews with four clinicians.
  arXiv  Detail & Related papers  (2021-08-04T17:34:13Z)
• Benchmarking Automated Clinical Language Simplification: Dataset, Algorithm, and Evaluation [48.87254340298189]
  We construct a new dataset named MedLane to support the development and evaluation of automated clinical language simplification approaches. We propose a new model called DECLARE that follows the human annotation procedure and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-12-04T06:09:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.