Why does my medical AI look at pictures of birds? Exploring the efficacy of transfer learning across domain boundaries

• URL: http://arxiv.org/abs/2306.17555v1
• Date: Fri, 30 Jun 2023 11:15:26 GMT
• Title: Why does my medical AI look at pictures of birds? Exploring the efficacy of transfer learning across domain boundaries
• Authors: Frederic Jonske, Moon Kim, Enrico Nasca, Janis Evers, Johannes Haubold, Ren\'e Hosch, Felix Nensa, Michael Kamp, Constantin Seibold, Jan Egger, Jens Kleesiek
• Abstract summary: Pretraining on data from the domain of the downstream task should almost always be preferred instead of ImageNet-pretrained models. We leverage RadNet-12M, a dataset containing more than 12 million computed tomography (CT) image slices. Our experiments cover intra- and cross-domain transfer scenarios, varying data scales, finetuning vs. linear evaluation, and feature space analysis.
• Score: 3.259272366439713
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: It is an open secret that ImageNet is treated as the panacea of pretraining. Particularly in medical machine learning, models not trained from scratch are often finetuned based on ImageNet-pretrained models. We posit that pretraining on data from the domain of the downstream task should almost always be preferred instead. We leverage RadNet-12M, a dataset containing more than 12 million computed tomography (CT) image slices, to explore the efficacy of self-supervised pretraining on medical and natural images. Our experiments cover intra- and cross-domain transfer scenarios, varying data scales, finetuning vs. linear evaluation, and feature space analysis. We observe that intra-domain transfer compares favorably to cross-domain transfer, achieving comparable or improved performance (0.44% - 2.07% performance increase using RadNet pretraining, depending on the experiment) and demonstrate the existence of a domain boundary-related generalization gap and domain-specific learned features.

Related papers

• MoCo-Transfer: Investigating out-of-distribution contrastive learning for limited-data domains [52.612507614610244]
  We analyze the benefit of transferring self-supervised contrastive representations from moment contrast (MoCo) pretraining to settings with limited data. We find that depending on quantity of labeled and unlabeled data, contrastive pretraining on larger out-of-distribution datasets can perform nearly as well or better than MoCo pretraining in-domain.
  arXiv  Detail & Related papers  (2023-11-15T21:56:47Z)
• Performance of GAN-based augmentation for deep learning COVID-19 image classification [57.1795052451257]
  The biggest challenge in the application of deep learning to the medical domain is the availability of training data. Data augmentation is a typical methodology used in machine learning when confronted with a limited data set. In this work, a StyleGAN2-ADA model of Generative Adversarial Networks is trained on the limited COVID-19 chest X-ray image set.
  arXiv  Detail & Related papers  (2023-04-18T15:39:58Z)
• Revisiting Hidden Representations in Transfer Learning for Medical Imaging [2.4545492329339815]
  We compare ImageNet and RadImageNet on seven medical classification tasks. Our results indicate that, contrary to intuition, ImageNet and RadImageNet may converge to distinct intermediate representations. Our findings show that the similarity between networks before and after fine-tuning does not correlate with performance gains.
  arXiv  Detail & Related papers  (2023-02-16T13:04:59Z)
• Using Language to Extend to Unseen Domains [81.37175826824625]
  It is expensive to collect training data for every possible domain that a vision model may encounter when deployed. We consider how simply verbalizing the training domain as well as domains we want to extend to but do not have data for can improve robustness. Using a multimodal model with a joint image and language embedding space, our method LADS learns a transformation of the image embeddings from the training domain to each unseen test domain.
  arXiv  Detail & Related papers  (2022-10-18T01:14:02Z)
• Learning from few examples: Classifying sex from retinal images via deep learning [3.9146761527401424]
  We showcase results for the performance of DL on small datasets to classify patient sex from fundus images. Our models, developed using approximately 2500 fundus images, achieved test AUC scores of up to 0.72. This corresponds to a mere 25% decrease in performance despite a nearly 1000-fold decrease in the dataset size.
  arXiv  Detail & Related papers  (2022-07-20T02:47:29Z)
• Semantic-aware Dense Representation Learning for Remote Sensing Image Change Detection [20.761672725633936]
  Training deep learning-based change detection model heavily depends on labeled data. Recent trend is using remote sensing (RS) data to obtain in-domain representations via supervised or self-supervised learning (SSL) We propose dense semantic-aware pre-training for RS image CD via sampling multiple class-balanced points.
  arXiv  Detail & Related papers  (2022-05-27T06:08:33Z)
• Unsupervised Domain Adaptation with Contrastive Learning for OCT Segmentation [49.59567529191423]
  We propose a novel semi-supervised learning framework for segmentation of volumetric images from new unlabeled domains. We jointly use supervised and contrastive learning, also introducing a contrastive pairing scheme that leverages similarity between nearby slices in 3D.
  arXiv  Detail & Related papers  (2022-03-07T19:02:26Z)
• Self-Supervised Pre-Training for Transformer-Based Person Re-Identification [54.55281692768765]
  Transformer-based supervised pre-training achieves great performance in person re-identification (ReID) Due to the domain gap between ImageNet and ReID datasets, it usually needs a larger pre-training dataset to boost the performance. This work aims to mitigate the gap between the pre-training and ReID datasets from the perspective of data and model structure.
  arXiv  Detail & Related papers  (2021-11-23T18:59:08Z)
• Pre-training Graph Neural Network for Cross Domain Recommendation [58.057687253631826]
  A recommender system predicts users' potential interests in items, where the core is to learn user/item embeddings. Inspired by the contemporary arts in pre-training from graph representation learning, we propose a pre-training and fine-tuning diagram for cross-domain recommendation. We devise a novel Pre-training Graph Neural Network for Cross-Domain Recommendation (PCRec), which adopts the contrastive self-supervised pre-training of a graph encoder.
  arXiv  Detail & Related papers  (2021-11-16T07:34:42Z)
• A Systematic Benchmarking Analysis of Transfer Learning for Medical Image Analysis [7.339428207644444]
  We conduct a systematic study on the transferability of models pre-trained on iNat2021, the most recent large-scale fine-grained dataset. We present a practical approach to bridge the domain gap between natural and medical images by continually (pre-training) supervised ImageNet models on medical images.
  arXiv  Detail & Related papers  (2021-08-12T19:08:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.