Automating Adjudication of Cardiovascular Events Using Large Language Models

• URL: http://arxiv.org/abs/2503.17222v1
• Date: Fri, 21 Mar 2025 15:25:53 GMT
• Title: Automating Adjudication of Cardiovascular Events Using Large Language Models
• Authors: Sonish Sivarajkumar, Kimia Ameri, Chuqin Li, Yanshan Wang, Min Jiang,
• Abstract summary: We present a novel framework for automating the adjudication of cardiovascular events in clinical trials using Large Language Models (LLMs)<n>Using cardiovascular event-specific clinical trial data, the framework achieved an F1-score of 0.82 for event extraction and an accuracy of 0.68 for adjudication.<n>This approach demonstrates significant potential for substantially reducing adjudication time and costs while maintaining high-quality, consistent, and auditable outcomes in clinical trials.
• Score: 3.7312896556790855
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cardiovascular events, such as heart attacks and strokes, remain a leading cause of mortality globally, necessitating meticulous monitoring and adjudication in clinical trials. This process, traditionally performed manually by clinical experts, is time-consuming, resource-intensive, and prone to inter-reviewer variability, potentially introducing bias and hindering trial progress. This study addresses these critical limitations by presenting a novel framework for automating the adjudication of cardiovascular events in clinical trials using Large Language Models (LLMs). We developed a two-stage approach: first, employing an LLM-based pipeline for event information extraction from unstructured clinical data and second, using an LLM-based adjudication process guided by a Tree of Thoughts approach and clinical endpoint committee (CEC) guidelines. Using cardiovascular event-specific clinical trial data, the framework achieved an F1-score of 0.82 for event extraction and an accuracy of 0.68 for adjudication. Furthermore, we introduce the CLEART score, a novel, automated metric specifically designed for evaluating the quality of AI-generated clinical reasoning in adjudicating cardiovascular events. This approach demonstrates significant potential for substantially reducing adjudication time and costs while maintaining high-quality, consistent, and auditable outcomes in clinical trials. The reduced variability and enhanced standardization also allow for faster identification and mitigation of risks associated with cardiovascular therapies.

Related papers

• FedCVD: The First Real-World Federated Learning Benchmark on Cardiovascular Disease Data [52.55123685248105]
  Cardiovascular diseases (CVDs) are currently the leading cause of death worldwide, highlighting the critical need for early diagnosis and treatment. Machine learning (ML) methods can help diagnose CVDs early, but their performance relies on access to substantial data with high quality. This paper presents the first real-world FL benchmark for cardiovascular disease detection, named FedCVD.
  arXiv  Detail & Related papers  (2024-10-28T02:24:01Z)
• Establishing Rigorous and Cost-effective Clinical Trials for Artificial Intelligence Models [18.240773244542474]
  A profound gap persists between artificial intelligence (AI) and clinical practice in medicine. State-of-the-art and state-of-the-practice AI model evaluations are limited to laboratory studies on medical datasets or direct clinical trials with no or solely patient-centered controls. For the first time, we emphasize the critical necessity for rigorous and cost-effective evaluation methodologies for AI models in clinical practice.
  arXiv  Detail & Related papers  (2024-07-11T14:37:08Z)
• TrialBench: Multi-Modal Artificial Intelligence-Ready Clinical Trial Datasets [57.067409211231244]
  This paper presents meticulously curated AIready datasets covering multi-modal data (e.g., drug molecule, disease code, text, categorical/numerical features) and 8 crucial prediction challenges in clinical trial design. We provide basic validation methods for each task to ensure the datasets' usability and reliability. We anticipate that the availability of such open-access datasets will catalyze the development of advanced AI approaches for clinical trial design.
  arXiv  Detail & Related papers  (2024-06-30T09:13:10Z)
• Language Interaction Network for Clinical Trial Approval Estimation [37.60098683485169]
  We introduce the Language Interaction Network (LINT), a novel approach that predicts trial outcomes using only the free-text descriptions of the trials. We have rigorously tested LINT across three phases of clinical trials, where it achieved ROC-AUC scores of 0.770, 0.740, and 0.748.
  arXiv  Detail & Related papers  (2024-04-26T14:50:59Z)
• TrialDura: Hierarchical Attention Transformer for Interpretable Clinical Trial Duration Prediction [19.084936647082632]
  We propose TrialDura, a machine learning-based method that estimates the duration of clinical trials using multimodal data. We encode them into Bio-BERT embeddings specifically tuned for biomedical contexts to provide a deeper and more relevant semantic understanding. Our proposed model demonstrated superior performance with a mean absolute error (MAE) of 1.04 years and a root mean square error (RMSE) of 1.39 years compared to the other models.
  arXiv  Detail & Related papers  (2024-04-20T02:12:59Z)
• AutoTrial: Prompting Language Models for Clinical Trial Design [53.630479619856516]
  We present a method named AutoTrial to aid the design of clinical eligibility criteria using language models. Experiments on over 70K clinical trials verify that AutoTrial generates high-quality criteria texts.
  arXiv  Detail & Related papers  (2023-05-19T01:04:16Z)
• Machine Learning to Support Triage of Children at Risk for Epileptic Seizures in the Pediatric Intensive Care Unit [5.708335717084799]
  Epileptic seizures are relatively common in critically-ill children admitted to the pediatric intensive care unit (PICU) Children deemed at risk for seizures within the PICU are monitored using continuous-electroencephalogram (cEEG) This research aims to develop a computer aided tool to improve seizures risk assessment in critically-ill children.
  arXiv  Detail & Related papers  (2022-05-11T10:24:58Z)
• Clinical Temporal Relation Extraction with Probabilistic Soft Logic Regularization and Global Inference [50.029659413650194]
  Existing methods either require expensive feature engineering or are incapable of modeling the global dependencies among the events. In this paper, we propose a novel method, Clinical Temporal ReLation Exaction with Probabilistic Soft Logic Regularization and Global Inference.
  arXiv  Detail & Related papers  (2020-12-16T08:23:03Z)
• MIA-Prognosis: A Deep Learning Framework to Predict Therapy Response [58.0291320452122]
  This paper aims at a unified deep learning approach to predict patient prognosis and therapy response. We formalize the prognosis modeling as a multi-modal asynchronous time series classification task. Our predictive model could further stratify low-risk and high-risk patients in terms of long-term survival.
  arXiv  Detail & Related papers  (2020-10-08T15:30:17Z)
• Prediction of the onset of cardiovascular diseases from electronic health records using multi-task gated recurrent units [51.14334174570822]
  We propose a multi-task recurrent neural network with attention mechanism for predicting cardiovascular events from electronic health records. The proposed approach is compared to a standard clinical risk predictor (QRISK) and machine learning alternatives using 5-year data from a NHS Foundation Trust.
  arXiv  Detail & Related papers  (2020-07-16T17:43:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.