Overcoming Data Scarcity in Biomedical Imaging with a Foundational Multi-Task Model

• URL: http://arxiv.org/abs/2311.09847v1
• Date: Thu, 16 Nov 2023 12:20:25 GMT
• Title: Overcoming Data Scarcity in Biomedical Imaging with a Foundational Multi-Task Model
• Authors: Raphael Sch\"afer, Till Nicke, Henning H\"ofener, Annkristin Lange, Dorit Merhof, Friedrich Feuerhake, Volkmar Schulz, Johannes Lotz, Fabian Kiessling
• Abstract summary: Foundational models, pretrained on a large scale, have demonstrated substantial success across non-medical domains. Here, we propose a multi-task learning strategy that decouples the number of training tasks from memory requirements.
• Score: 2.5994154212235685
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Foundational models, pretrained on a large scale, have demonstrated substantial success across non-medical domains. However, training these models typically requires large, comprehensive datasets, which contrasts with the smaller and more heterogeneous datasets common in biomedical imaging. Here, we propose a multi-task learning strategy that decouples the number of training tasks from memory requirements. We trained a Universal bioMedical PreTrained model (UMedPT) on a multi-task database including tomographic, microscopic, and X-ray images, with various labelling strategies such as classification, segmentation, and object detection. The UMedPT foundational model outperformed ImageNet pretraining and the previous state-of-the-art models. For tasks related to the pretraining database, it maintained its performance with only 1% of the original training data and without fine-tuning. For out-of-domain tasks it required not more than 50% of the original training data. In an external independent validation imaging features extracted using UMedPT proved to be a new standard for cross-center transferability.

Related papers

• MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining [73.81862342673894]
  Foundation models have reshaped the landscape of Remote Sensing (RS) by enhancing various image interpretation tasks. transferring the pretrained models to downstream tasks may encounter task discrepancy due to their formulation of pretraining as image classification or object discrimination tasks. We conduct multi-task supervised pretraining on the SAMRS dataset, encompassing semantic segmentation, instance segmentation, and rotated object detection. Our models are finetuned on various RS downstream tasks, such as scene classification, horizontal and rotated object detection, semantic segmentation, and change detection.
  arXiv  Detail & Related papers  (2024-03-20T09:17:22Z)
• VISION-MAE: A Foundation Model for Medical Image Segmentation and Classification [36.8105960525233]
  We present a novel foundation model, VISION-MAE, specifically designed for medical imaging. VISION-MAE is trained on a dataset of 2.5 million unlabeled images from various modalities. It is then adapted to classification and segmentation tasks using explicit labels.
  arXiv  Detail & Related papers  (2024-02-01T21:45:12Z)
• Building Universal Foundation Models for Medical Image Analysis with Spatially Adaptive Networks [5.661631789478932]
  We propose a universal foundation model for medical image analysis that processes images with heterogeneous spatial properties using a unified structure. We pre-train a spatial adaptive visual tokenizer (SPAD-VT) and then a spatial adaptive Vision Transformer (SPAD-ViT) via masked image modeling (MIM) on 55 public medical image datasets. The experimental results on downstream medical image classification and segmentation tasks demonstrate the superior performance and label efficiency of our model.
  arXiv  Detail & Related papers  (2023-12-12T08:33:45Z)
• An Efficient General-Purpose Modular Vision Model via Multi-Task Heterogeneous Training [79.78201886156513]
  We present a model that can perform multiple vision tasks and can be adapted to other downstream tasks efficiently. Our approach achieves comparable results to single-task state-of-the-art models and demonstrates strong generalization on downstream tasks.
  arXiv  Detail & Related papers  (2023-06-29T17:59:57Z)
• LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching [59.01894976615714]
  We introduce LVM-Med, the first family of deep networks trained on large-scale medical datasets. We have collected approximately 1.3 million medical images from 55 publicly available datasets. LVM-Med empirically outperforms a number of state-of-the-art supervised, self-supervised, and foundation models.
  arXiv  Detail & Related papers  (2023-06-20T22:21:34Z)
• Learnable Weight Initialization for Volumetric Medical Image Segmentation [66.3030435676252]
  We propose a learnable weight-based hybrid medical image segmentation approach. Our approach is easy to integrate into any hybrid model and requires no external training data. Experiments on multi-organ and lung cancer segmentation tasks demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2023-06-15T17:55:05Z)
• Domain Generalization for Mammographic Image Analysis with Contrastive Learning [62.25104935889111]
  The training of an efficacious deep learning model requires large data with diverse styles and qualities. A novel contrastive learning is developed to equip the deep learning models with better style generalization capability. The proposed method has been evaluated extensively and rigorously with mammograms from various vendor style domains and several public datasets.
  arXiv  Detail & Related papers  (2023-04-20T11:40:21Z)
• MultiTalent: A Multi-Dataset Approach to Medical Image Segmentation [1.146419670457951]
  Current practices limit model training and supervised pre-training to one or a few similar datasets. We propose MultiTalent, a method that leverages multiple CT datasets with diverse and conflicting class definitions.
  arXiv  Detail & Related papers  (2023-03-25T11:37:16Z)
• The effectiveness of MAE pre-pretraining for billion-scale pretraining [65.98338857597935]
  We introduce an additional pre-pretraining stage that is simple and uses the self-supervised MAE technique to initialize the model. We measure the effectiveness of pre-pretraining on 10 different visual recognition tasks spanning image classification, video recognition, object detection, low-shot classification and zero-shot recognition.
  arXiv  Detail & Related papers  (2023-03-23T17:56:12Z)
• Generalized Multi-Task Learning from Substantially Unlabeled Multi-Source Medical Image Data [11.061381376559053]
  MultiMix is a new multi-task learning model that jointly learns disease classification and anatomical segmentation in a semi-supervised manner. Our experiments with varying quantities of multi-source labeled data in the training sets confirm the effectiveness of MultiMix.
  arXiv  Detail & Related papers  (2021-10-25T18:09:19Z)
• Multi-task pre-training of deep neural networks for digital pathology [8.74883469030132]
  We first assemble and transform many digital pathology datasets into a pool of 22 classification tasks and almost 900k images. We show that our models used as feature extractors either improve significantly over ImageNet pre-trained models or provide comparable performance.
  arXiv  Detail & Related papers  (2020-05-05T08:50:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.