Assessment of contextualised representations in detecting outcome
phrases in clinical trials
- URL: http://arxiv.org/abs/2203.03547v1
- Date: Sun, 13 Feb 2022 15:08:00 GMT
- Title: Assessment of contextualised representations in detecting outcome
phrases in clinical trials
- Authors: Micheal Abaho, Danushka Bollegala, Paula R Williamson, Susanna Dodd
- Abstract summary: We introduce "EBM-COMET", a dataset in which 300 PubMed abstracts are expertly annotated for clinical outcomes.
To extract outcomes, we fine-tune a variety of pre-trained contextualized representations.
We observe our best model (BioBERT) achieve 81.5% F1, 81.3% sensitivity and 98.0% specificity.
- Score: 14.584741378279316
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Automating the recognition of outcomes reported in clinical trials using
machine learning has a huge potential of speeding up access to evidence
necessary in healthcare decision-making. Prior research has however
acknowledged inadequate training corpora as a challenge for the Outcome
detection (OD) task. Additionally, several contextualized representations like
BERT and ELMO have achieved unparalleled success in detecting various diseases,
genes, proteins, and chemicals, however, the same cannot be emphatically stated
for outcomes, because these models have been relatively under-tested and
studied for the OD task. We introduce "EBM-COMET", a dataset in which 300
PubMed abstracts are expertly annotated for clinical outcomes. Unlike prior
related datasets that use arbitrary outcome classifications, we use labels from
a taxonomy recently published to standardize outcome classifications. To
extract outcomes, we fine-tune a variety of pre-trained contextualized
representations, additionally, we use frozen contextualized and
context-independent representations in our custom neural model augmented with
clinically informed Part-Of-Speech embeddings and a cost-sensitive loss
function. We adopt strict evaluation for the trained models by rewarding them
for correctly identifying full outcome phrases rather than words within the
entities i.e. given an outcome "systolic blood pressure", the models are
rewarded a classification score only when they predict all 3 words in sequence,
otherwise, they are not rewarded. We observe our best model (BioBERT) achieve
81.5\% F1, 81.3\% sensitivity and 98.0\% specificity. We reach a consensus on
which contextualized representations are best suited for detecting outcomes
from clinical-trial abstracts. Furthermore, our best model outperforms scores
published on the original EBM-NLP dataset leader-board scores.
Related papers
- Towards Efficient Patient Recruitment for Clinical Trials: Application of a Prompt-Based Learning Model [0.7373617024876725]
Clinical trials are essential for advancing pharmaceutical interventions, but they face a bottleneck in selecting eligible participants.
The complex nature of unstructured medical texts presents challenges in efficiently identifying participants.
In this study, we aimed to evaluate the performance of a prompt-based large language model for the cohort selection task.
arXiv Detail & Related papers (2024-04-24T20:42:28Z) - Uncertainty Quantification on Clinical Trial Outcome Prediction [37.238845949535616]
We propose incorporating uncertainty quantification into clinical trial outcome predictions.
Our main goal is to enhance the model's ability to discern nuanced differences.
We have adopted a selective classification approach to fulfill our objective.
arXiv Detail & Related papers (2024-01-07T13:48:05Z) - Multi-task Explainable Skin Lesion Classification [54.76511683427566]
We propose a few-shot-based approach for skin lesions that generalizes well with few labelled data.
The proposed approach comprises a fusion of a segmentation network that acts as an attention module and classification network.
arXiv Detail & Related papers (2023-10-11T05:49:47Z) - TREEMENT: Interpretable Patient-Trial Matching via Personalized Dynamic
Tree-Based Memory Network [54.332862955411656]
Clinical trials are critical for drug development but often suffer from expensive and inefficient patient recruitment.
In recent years, machine learning models have been proposed for speeding up patient recruitment via automatically matching patients with clinical trials.
We introduce a dynamic tree-based memory network model named TREEMENT to provide accurate and interpretable patient trial matching.
arXiv Detail & Related papers (2023-07-19T12:35:09Z) - SPOT: Sequential Predictive Modeling of Clinical Trial Outcome with
Meta-Learning [67.8195828626489]
Clinical trials are essential to drug development but time-consuming, costly, and prone to failure.
We propose Sequential Predictive mOdeling of clinical Trial outcome (SPOT) that first identifies trial topics to cluster the multi-sourced trial data into relevant trial topics.
With the consideration of each trial sequence as a task, it uses a meta-learning strategy to achieve a point where the model can rapidly adapt to new tasks with minimal updates.
arXiv Detail & Related papers (2023-04-07T23:04:27Z) - Avoiding Biased Clinical Machine Learning Model Performance Estimates in
the Presence of Label Selection [3.3944964838781093]
We describe three classes of label selection and simulate five causally distinct scenarios to assess how particular selection mechanisms bias a suite of commonly reported binary machine learning model performance metrics.
We find that naive estimates of AUROC on the observed population undershoot actual performance by up to 20%.
Such a disparity could be large enough to lead to the wrongful termination of a successful clinical decision support tool.
arXiv Detail & Related papers (2022-09-15T22:30:14Z) - Negation detection in Dutch clinical texts: an evaluation of rule-based
and machine learning methods [0.21079694661943607]
We compare three methods for negation detection in Dutch clinical notes.
We found that both the biLSTM and RoBERTa models consistently outperform the rule-based model in terms of F1 score, precision and recall.
arXiv Detail & Related papers (2022-09-01T14:00:13Z) - Estimating and Improving Fairness with Adversarial Learning [65.99330614802388]
We propose an adversarial multi-task training strategy to simultaneously mitigate and detect bias in the deep learning-based medical image analysis system.
Specifically, we propose to add a discrimination module against bias and a critical module that predicts unfairness within the base classification model.
We evaluate our framework on a large-scale public-available skin lesion dataset.
arXiv Detail & Related papers (2021-03-07T03:10:32Z) - Understanding Clinical Trial Reports: Extracting Medical Entities and
Their Relations [33.30381080306156]
Medical experts must manually extract information from articles to inform decision-making.
We consider the end-to-end task of both (a) extracting treatments and outcomes from full-text articles describing clinical trials (entity identification) and (b) inferring the reported results for the former with respect to the latter.
arXiv Detail & Related papers (2020-10-07T17:50:58Z) - Semi-supervised Medical Image Classification with Relation-driven
Self-ensembling Model [71.80319052891817]
We present a relation-driven semi-supervised framework for medical image classification.
It exploits the unlabeled data by encouraging the prediction consistency of given input under perturbations.
Our method outperforms many state-of-the-art semi-supervised learning methods on both single-label and multi-label image classification scenarios.
arXiv Detail & Related papers (2020-05-15T06:57:54Z) - Hemogram Data as a Tool for Decision-making in COVID-19 Management:
Applications to Resource Scarcity Scenarios [62.997667081978825]
COVID-19 pandemics has challenged emergency response systems worldwide, with widespread reports of essential services breakdown and collapse of health care structure.
This work describes a machine learning model derived from hemogram exam data performed in symptomatic patients.
Proposed models can predict COVID-19 qRT-PCR results in symptomatic individuals with high accuracy, sensitivity and specificity.
arXiv Detail & Related papers (2020-05-10T01:45:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.