Ranking pre-trained segmentation models for zero-shot transferability

• URL: http://arxiv.org/abs/2503.00450v1
• Date: Sat, 01 Mar 2025 11:11:06 GMT
• Title: Ranking pre-trained segmentation models for zero-shot transferability
• Authors: Joshua Talks, Anna Kreshuk,
• Abstract summary: Huge cost of labelling sufficient training data is a major bottleneck in the use of deep learning.<n>We propose the first unsupervised transferability estimator for semantic and instance segmentation tasks.<n>We evaluate the method on multiple segmentation problems across microscopy modalities.
• Score: 3.0496043297705424
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Model transfer presents a solution to the challenges of segmentation in the microscopy community, where the immense cost of labelling sufficient training data is a major bottleneck in the use of deep learning. With large quantities of imaging data produced across a wide range of imaging conditions, institutes also produce many bespoke models trained on specific source data which then get collected in model banks or zoos. As the number of available models grows, so does the need for an efficient and reliable model selection method for a specific target dataset of interest. We focus on the unsupervised regime where no labels are available for the target dataset. Building on previous work linking model generalisation and consistency under perturbation, we propose the first unsupervised transferability estimator for semantic and instance segmentation tasks which doesn't require access to source training data or target domain labels. We evaluate the method on multiple segmentation problems across microscopy modalities, finding a strong correlation between the rankings based on our estimator and rankings based on target dataset performance.

Related papers

• A Two-Phase Recall-and-Select Framework for Fast Model Selection [13.385915962994806]
  We propose a two-phase (coarse-recall and fine-selection) model selection framework. It aims to enhance the efficiency of selecting a robust model by leveraging the models' training performances on benchmark datasets. It has been demonstrated that the proposed methodology facilitates the selection of a high-performing model at a rate about 3x times faster than conventional baseline methods.
  arXiv  Detail & Related papers  (2024-03-28T14:44:44Z)
• DsDm: Model-Aware Dataset Selection with Datamodels [81.01744199870043]
  Standard practice is to filter for examples that match human notions of data quality. We find that selecting according to similarity with "high quality" data sources may not increase (and can even hurt) performance compared to randomly selecting data. Our framework avoids handpicked notions of data quality, and instead models explicitly how the learning process uses train datapoints to predict on the target tasks.
  arXiv  Detail & Related papers  (2024-01-23T17:22:00Z)
• Cross-Level Distillation and Feature Denoising for Cross-Domain Few-Shot Classification [49.36348058247138]
  We tackle the problem of cross-domain few-shot classification by making a small proportion of unlabeled images in the target domain accessible in the training stage. We meticulously design a cross-level knowledge distillation method, which can strengthen the ability of the model to extract more discriminative features in the target dataset. Our approach can surpass the previous state-of-the-art method, Dynamic-Distillation, by 5.44% on 1-shot and 1.37% on 5-shot classification tasks.
  arXiv  Detail & Related papers  (2023-11-04T12:28:04Z)
• Building a Winning Team: Selecting Source Model Ensembles using a Submodular Transferability Estimation Approach [20.86345962679122]
  Estimating the transferability of publicly available pretrained models to a target task has assumed an important place for transfer learning tasks. We propose a novel Optimal tranSport-based suBmOdular tRaNsferability metric (OSBORN) to estimate the transferability of an ensemble of models to a downstream task.
  arXiv  Detail & Related papers  (2023-09-05T17:57:31Z)
• Universal Semi-supervised Model Adaptation via Collaborative Consistency Training [92.52892510093037]
  We introduce a realistic and challenging domain adaptation problem called Universal Semi-supervised Model Adaptation (USMA) We propose a collaborative consistency training framework that regularizes the prediction consistency between two models. Experimental results demonstrate the effectiveness of our method on several benchmark datasets.
  arXiv  Detail & Related papers  (2023-07-07T08:19:40Z)
• Self-Distillation for Further Pre-training of Transformers [83.84227016847096]
  We propose self-distillation as a regularization for a further pre-training stage. We empirically validate the efficacy of self-distillation on a variety of benchmark datasets for image and text classification tasks.
  arXiv  Detail & Related papers  (2022-09-30T02:25:12Z)
• Semi-supervised Deep Learning for Image Classification with Distribution Mismatch: A Survey [1.5469452301122175]
  Deep learning models rely on the abundance of labelled observations to train a prospective model. It is expensive to gather labelled observations of data, making the usage of deep learning models not ideal. In many situations different unlabelled data sources might be available. This raises the risk of a significant distribution mismatch between the labelled and unlabelled datasets.
  arXiv  Detail & Related papers  (2022-03-01T02:46:00Z)
• MSeg: A Composite Dataset for Multi-domain Semantic Segmentation [100.17755160696939]
  We present MSeg, a composite dataset that unifies semantic segmentation datasets from different domains. We reconcile the generalization and bring the pixel-level annotations into alignment by relabeling more than 220,000 object masks in more than 80,000 images. A model trained on MSeg ranks first on the WildDash-v1 leaderboard for robust semantic segmentation, with no exposure to WildDash data during training.
  arXiv  Detail & Related papers  (2021-12-27T16:16:35Z)
• Transferability Estimation using Bhattacharyya Class Separability [37.52588126267552]
  Transfer learning is a popular method for leveraging pre-trained models in computer vision. It is difficult to quantify which pre-trained source models are suitable for a specific target task. We propose a novel method for quantifying transferability between a source model and a target dataset.
  arXiv  Detail & Related papers  (2021-11-24T20:22:28Z)
• A Survey on Self-supervised Pre-training for Sequential Transfer Learning in Neural Networks [1.1802674324027231]
  Self-supervised pre-training for transfer learning is becoming an increasingly popular technique to improve state-of-the-art results using unlabeled data. We provide an overview of the taxonomy for self-supervised learning and transfer learning, and highlight some prominent methods for designing pre-training tasks across different domains.
  arXiv  Detail & Related papers  (2020-07-01T22:55:48Z)
• Uncertainty-aware Self-training for Text Classification with Few Labels [54.13279574908808]
  We study self-training as one of the earliest semi-supervised learning approaches to reduce the annotation bottleneck. We propose an approach to improve self-training by incorporating uncertainty estimates of the underlying neural network. We show our methods leveraging only 20-30 labeled samples per class for each task for training and for validation can perform within 3% of fully supervised pre-trained language models.
  arXiv  Detail & Related papers  (2020-06-27T08:13:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.