CheX-GPT: Harnessing Large Language Models for Enhanced Chest X-ray
Report Labeling
- URL: http://arxiv.org/abs/2401.11505v1
- Date: Sun, 21 Jan 2024 14:30:20 GMT
- Title: CheX-GPT: Harnessing Large Language Models for Enhanced Chest X-ray
Report Labeling
- Authors: Jawook Gu, Han-Cheol Cho, Jiho Kim, Kihyun You, Eun Kyoung Hong,
Byungseok Roh
- Abstract summary: Free-text radiology reports present a rich data source for various medical tasks, but effectively labeling these texts remains challenging.
Traditional rule-based labeling methods fall short capturing the nuances of diverse free-text patterns.
Our study offers three main contributions: 1) We demonstrate the potential of GPT as an adept labeler using carefully designed prompts; 2) We trained a BERT-based labeler, CheX-GPT, which operates faster and more efficiently than its GPT counterpart; and 3) To benchmark labeler performance, we introduced a publicly available expert-annotated test set, MIMIC-500.
- Score: 7.219847880678653
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Free-text radiology reports present a rich data source for various medical
tasks, but effectively labeling these texts remains challenging. Traditional
rule-based labeling methods fall short of capturing the nuances of diverse
free-text patterns. Moreover, models using expert-annotated data are limited by
data scarcity and pre-defined classes, impacting their performance, flexibility
and scalability. To address these issues, our study offers three main
contributions: 1) We demonstrate the potential of GPT as an adept labeler using
carefully designed prompts. 2) Utilizing only the data labeled by GPT, we
trained a BERT-based labeler, CheX-GPT, which operates faster and more
efficiently than its GPT counterpart. 3) To benchmark labeler performance, we
introduced a publicly available expert-annotated test set, MIMIC-500,
comprising 500 cases from the MIMIC validation set. Our findings demonstrate
that CheX-GPT not only excels in labeling accuracy over existing models, but
also showcases superior efficiency, flexibility, and scalability, supported by
our introduction of the MIMIC-500 dataset for robust benchmarking. Code and
models are available at https://github.com/kakaobrain/CheXGPT.
Related papers
- Enhancing chest X-ray datasets with privacy-preserving large language
models and multi-type annotations: a data-driven approach for improved
classification [0.6906005491572398]
In chest X-ray (CXR) image analysis, rule-based systems are usually employed to extract labels from reports.
We present MAPLEZ, a novel approach leveraging a locally executable Large Language Model (LLM) to extract and enhance findings labels on CXR reports.
arXiv Detail & Related papers (2024-03-06T20:10:41Z) - Investigating Large Language Models and Control Mechanisms to Improve Text Readability of Biomedical Abstracts [16.05119302860606]
We investigate the ability of state-of-the-art large language models (LLMs) on the task of biomedical abstract simplification.
The methods applied include domain fine-tuning and prompt-based learning (PBL)
We used a range of automatic evaluation metrics, including BLEU, ROUGE, SARI, and BERTscore, and also conducted human evaluations.
arXiv Detail & Related papers (2023-09-22T22:47:32Z) - SLPT: Selective Labeling Meets Prompt Tuning on Label-Limited Lesion
Segmentation [57.37875162629063]
We propose a framework that combines selective labeling with prompt tuning to boost performance in limited labels.
We evaluate our method on liver tumor segmentation and achieve state-of-the-art performance, outperforming traditional fine-tuning with only 6% of tunable parameters.
arXiv Detail & Related papers (2023-08-09T12:22:49Z) - Automated Labeling of German Chest X-Ray Radiology Reports using Deep
Learning [50.591267188664666]
We propose a deep learning-based CheXpert label prediction model, pre-trained on reports labeled by a rule-based German CheXpert model.
Our results demonstrate the effectiveness of our approach, which significantly outperformed the rule-based model on all three tasks.
arXiv Detail & Related papers (2023-06-09T16:08:35Z) - Text-Augmented Open Knowledge Graph Completion via Pre-Trained Language
Models [53.09723678623779]
We propose TAGREAL to automatically generate quality query prompts and retrieve support information from large text corpora.
The results show that TAGREAL achieves state-of-the-art performance on two benchmark datasets.
We find that TAGREAL has superb performance even with limited training data, outperforming existing embedding-based, graph-based, and PLM-based methods.
arXiv Detail & Related papers (2023-05-24T22:09:35Z) - PINA: Leveraging Side Information in eXtreme Multi-label Classification
via Predicted Instance Neighborhood Aggregation [105.52660004082766]
The eXtreme Multi-label Classification(XMC) problem seeks to find relevant labels from an exceptionally large label space.
We propose Predicted Instance Neighborhood Aggregation (PINA), a data enhancement method for the general XMC problem.
Unlike most existing XMC frameworks that treat labels and input instances as featureless indicators and independent entries, PINA extracts information from the label metadata and the correlations among training instances.
arXiv Detail & Related papers (2023-05-21T05:00:40Z) - Ground Truth Inference for Weakly Supervised Entity Matching [76.6732856489872]
We propose a simple but powerful labeling model for weak supervision tasks.
We then tailor the labeling model specifically to the task of entity matching.
We show that our labeling model results in a 9% higher F1 score on average than the best existing method.
arXiv Detail & Related papers (2022-11-13T17:57:07Z) - Eliciting and Learning with Soft Labels from Every Annotator [31.10635260890126]
We focus on efficiently eliciting soft labels from individual annotators.
We demonstrate that learning with our labels achieves comparable model performance to prior approaches.
arXiv Detail & Related papers (2022-07-02T12:03:00Z) - Students Need More Attention: BERT-based AttentionModel for Small Data
with Application to AutomaticPatient Message Triage [65.7062363323781]
We propose a novel framework based on BioBERT (Bidirectional Representations from Transformers forBiomedical TextMining)
We introduce Label Embeddings for Self-Attention in each layer of BERT, which we call LESA-BERT, and (ii) by distilling LESA-BERT to smaller variants, we aim to reduce overfitting and model size when working on small datasets.
As an application, our framework is utilized to build a model for patient portal message triage that classifies the urgency of a message into three categories: non-urgent, medium and urgent.
arXiv Detail & Related papers (2020-06-22T03:39:00Z) - CheXbert: Combining Automatic Labelers and Expert Annotations for
Accurate Radiology Report Labeling Using BERT [6.458158112222296]
We introduce a BERT-based approach to medical image report labeling.
We demonstrate superior performance of a biomedically pretrained BERT model first trained on annotations of a rule-based labeler.
We find that our final model, CheXbert, is able to outperform the previous best rules-based labeler with statistical significance.
arXiv Detail & Related papers (2020-04-20T09:46:40Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.