Significantly improving zero-shot X-ray pathology classification via
fine-tuning pre-trained image-text encoders
- URL: http://arxiv.org/abs/2212.07050v1
- Date: Wed, 14 Dec 2022 06:04:18 GMT
- Title: Significantly improving zero-shot X-ray pathology classification via
fine-tuning pre-trained image-text encoders
- Authors: Jongseong Jang, Daeun Kyung, Seung Hwan Kim, Honglak Lee, Kyunghoon
Bae, Edward Choi
- Abstract summary: We propose a new fine-tuning strategy based on sentence sampling and positive-pair loss relaxation for improving the downstream zero-shot pathology classification performance.
Our method consistently showed dramatically improved zero-shot pathology classification performance on four different chest X-ray datasets.
- Score: 51.14431540035141
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Deep neural networks have been successfully adopted to diverse domains
including pathology classification based on medical images. However,
large-scale and high-quality data to train powerful neural networks are rare in
the medical domain as the labeling must be done by qualified experts.
Researchers recently tackled this problem with some success by taking advantage
of models pre-trained on large-scale general domain data. Specifically,
researchers took contrastive image-text encoders (e.g., CLIP) and fine-tuned it
with chest X-ray images and paired reports to perform zero-shot pathology
classification, thus completely removing the need for pathology-annotated
images to train a classification model. Existing studies, however, fine-tuned
the pre-trained model with the same contrastive learning objective, and failed
to exploit the multi-labeled nature of medical image-report pairs. In this
paper, we propose a new fine-tuning strategy based on sentence sampling and
positive-pair loss relaxation for improving the downstream zero-shot pathology
classification performance, which can be applied to any pre-trained contrastive
image-text encoders. Our method consistently showed dramatically improved
zero-shot pathology classification performance on four different chest X-ray
datasets and 3 different pre-trained models (5.77% average AUROC increase). In
particular, fine-tuning CLIP with our method showed much comparable or
marginally outperformed to board-certified radiologists (0.619 vs 0.625 in F1
score and 0.530 vs 0.544 in MCC) in zero-shot classification of five prominent
diseases from the CheXpert dataset.
Related papers
- Performance of GAN-based augmentation for deep learning COVID-19 image
classification [57.1795052451257]
The biggest challenge in the application of deep learning to the medical domain is the availability of training data.
Data augmentation is a typical methodology used in machine learning when confronted with a limited data set.
In this work, a StyleGAN2-ADA model of Generative Adversarial Networks is trained on the limited COVID-19 chest X-ray image set.
arXiv Detail & Related papers (2023-04-18T15:39:58Z) - Vision-Language Modelling For Radiological Imaging and Reports In The
Low Data Regime [70.04389979779195]
This paper explores training medical vision-language models (VLMs) where the visual and language inputs are embedded into a common space.
We explore several candidate methods to improve low-data performance, including adapting generic pre-trained models to novel image and text domains.
Using text-to-image retrieval as a benchmark, we evaluate the performance of these methods with variable sized training datasets of paired chest X-rays and radiological reports.
arXiv Detail & Related papers (2023-03-30T18:20:00Z) - COVID-19 Severity Classification on Chest X-ray Images [0.0]
In this work, we classify covid images based on the severity of the infection.
The ResNet-50 model produced remarkable classification results in terms of accuracy 95%, recall (0.94), and F1-Score (0.92), and precision (0.91)
arXiv Detail & Related papers (2022-05-25T12:01:03Z) - Application of Transfer Learning and Ensemble Learning in Image-level
Classification for Breast Histopathology [9.037868656840736]
In Computer-Aided Diagnosis (CAD), traditional classification models mostly use a single network to extract features.
This paper proposes a deep ensemble model based on image-level labels for the binary classification of benign and malignant lesions.
Result: In the ensemble network model with accuracy as the weight, the image-level binary classification achieves an accuracy of $98.90%$.
arXiv Detail & Related papers (2022-04-18T13:31:53Z) - Vision Transformers for femur fracture classification [59.99241204074268]
The Vision Transformer (ViT) was able to correctly predict 83% of the test images.
Good results were obtained in sub-fractures with the largest and richest dataset ever.
arXiv Detail & Related papers (2021-08-07T10:12:42Z) - Transfer Learning with Ensembles of Deep Neural Networks for Skin Cancer
Classification in Imbalanced Data Sets [0.6802401545890961]
Machine learning techniques for accurate classification of skin cancer from medical images have been reported.
Many techniques are based on pre-trained convolutional neural networks (CNNs), which enable training the models based on limited amounts of training data.
We propose a novel ensemble-based CNN architecture where multiple CNN models, some of which are pre-trained and some are trained only on the data at hand, along with patient information (meta-data) are combined using a meta-learner.
arXiv Detail & Related papers (2021-03-22T06:04:45Z) - Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for
Thoracic Disease Identification [83.6017225363714]
deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance.
For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming.
In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
arXiv Detail & Related papers (2021-02-26T02:29:30Z) - Metastatic Cancer Image Classification Based On Deep Learning Method [7.832709940526033]
We propose a noval method which combines the deep learning algorithm in image classification, the DenseNet169 framework and Rectified Adam optimization algorithm.
Our model achieves superior performance over the other classical convolutional neural networks approaches, such as Vgg19, Resnet34, Resnet50.
arXiv Detail & Related papers (2020-11-13T16:04:39Z) - A Deep Learning Study on Osteosarcoma Detection from Histological Images [6.341765152919201]
The most common type of primary malignant bone tumor is osteosarcoma.
CNNs can significantly decrease surgeon's workload and make a better prognosis of patient conditions.
CNNs need to be trained on a large amount of data in order to achieve a more trustworthy performance.
arXiv Detail & Related papers (2020-11-02T18:16:17Z) - Multi-label Thoracic Disease Image Classification with Cross-Attention
Networks [65.37531731899837]
We propose a novel scheme of Cross-Attention Networks (CAN) for automated thoracic disease classification from chest x-ray images.
We also design a new loss function that beyond cross-entropy loss to help cross-attention process and is able to overcome the imbalance between classes and easy-dominated samples within each class.
arXiv Detail & Related papers (2020-07-21T14:37:00Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.