Systematic comparison of semi-supervised and self-supervised learning for medical image classification

• URL: http://arxiv.org/abs/2307.08919v3
• Date: Fri, 29 Mar 2024 18:19:36 GMT
• Title: Systematic comparison of semi-supervised and self-supervised learning for medical image classification
• Authors: Zhe Huang, Ruijie Jiang, Shuchin Aeron, Michael C. Hughes,
• Abstract summary: In typical medical image classification problems, labeled data is scarce while unlabeled data is more available. Recent methods from both directions have reported significant gains on traditional benchmarks. Our study compares 13 representative semi- and self-supervised methods to strong labeled-set-only baselines on 4 medical datasets.
• Score: 12.977356726499735
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In typical medical image classification problems, labeled data is scarce while unlabeled data is more available. Semi-supervised learning and self-supervised learning are two different research directions that can improve accuracy by learning from extra unlabeled data. Recent methods from both directions have reported significant gains on traditional benchmarks. Yet past benchmarks do not focus on medical tasks and rarely compare self- and semi- methods together on an equal footing. Furthermore, past benchmarks often handle hyperparameter tuning suboptimally. First, they may not tune hyperparameters at all, leading to underfitting. Second, when tuning does occur, it often unrealistically uses a labeled validation set that is much larger than the training set. Therefore currently published rankings might not always corroborate with their practical utility This study contributes a systematic evaluation of self- and semi- methods with a unified experimental protocol intended to guide a practitioner with scarce overall labeled data and a limited compute budget. We answer two key questions: Can hyperparameter tuning be effective with realistic-sized validation sets? If so, when all methods are tuned well, which self- or semi-supervised methods achieve the best accuracy? Our study compares 13 representative semi- and self-supervised methods to strong labeled-set-only baselines on 4 medical datasets. From 20000+ GPU hours of computation, we provide valuable best practices to resource-constrained practitioners: hyperparameter tuning is effective, and the semi-supervised method known as MixMatch delivers the most reliable gains across 4 datasets.

Related papers

• Optimizing Resources for On-the-Fly Label Estimation with Multiple Unknown Medical Experts [2.904892426557913]
  We propose an adaptive approach for real-time annotation that supports on-the-fly labeling of incoming data.<n>We evaluate our approach on three multi-annotator classification datasets across different modalities.
  arXiv  Detail & Related papers  (2025-10-04T21:41:26Z)
• Probably Approximately Precision and Recall Learning [60.00180898830079]
  A key challenge in machine learning is the prevalence of one-sided feedback.<n>We introduce a Probably Approximately Correct (PAC) framework in which hypotheses are set functions that map each input to a set of labels.<n>We develop new algorithms that learn from positive data alone, achieving optimal sample complexity in the realizable case.
  arXiv  Detail & Related papers  (2024-11-20T04:21:07Z)
• Incremental Self-training for Semi-supervised Learning [56.57057576885672]
  IST is simple yet effective and fits existing self-training-based semi-supervised learning methods. We verify the proposed IST on five datasets and two types of backbone, effectively improving the recognition accuracy and learning speed.
  arXiv  Detail & Related papers  (2024-04-14T05:02:00Z)
• One-bit Supervision for Image Classification: Problem, Solution, and Beyond [114.95815360508395]
  This paper presents one-bit supervision, a novel setting of learning with fewer labels, for image classification. We propose a multi-stage training paradigm and incorporate negative label suppression into an off-the-shelf semi-supervised learning algorithm. In multiple benchmarks, the learning efficiency of the proposed approach surpasses that using full-bit, semi-supervised supervision.
  arXiv  Detail & Related papers  (2023-11-26T07:39:00Z)
• Dual-Decoder Consistency via Pseudo-Labels Guided Data Augmentation for Semi-Supervised Medical Image Segmentation [13.707121013895929]
  We present a novel semi-supervised learning method, Dual-Decoder Consistency via Pseudo-Labels Guided Data Augmentation. We use distinct decoders for student and teacher networks while maintain the same encoder. To learn from unlabeled data, we create pseudo-labels generated by the teacher networks and augment the training data with the pseudo-labels.
  arXiv  Detail & Related papers  (2023-08-31T09:13:34Z)
• Semi-Supervised Learning with Multiple Imputations on Non-Random Missing Labels [0.0]
  Semi-Supervised Learning (SSL) is implemented when algorithms are trained on both labeled and unlabeled data. This paper proposes two new methods of combining multiple imputation models to achieve higher accuracy and less bias.
  arXiv  Detail & Related papers  (2023-08-15T04:09:53Z)
• Enhancing Adversarial Robustness in Low-Label Regime via Adaptively Weighted Regularization and Knowledge Distillation [1.675857332621569]
  We investigate semi-supervised adversarial training where labeled data is scarce. We develop a semi-supervised adversarial training algorithm that combines the proposed regularization term with knowledge distillation. Our proposed algorithm achieves state-of-the-art performance with significant margins compared to existing algorithms.
  arXiv  Detail & Related papers  (2023-08-08T05:48:38Z)
• Debiased Pseudo Labeling in Self-Training [77.83549261035277]
  Deep neural networks achieve remarkable performances on a wide range of tasks with the aid of large-scale labeled datasets. To mitigate the requirement for labeled data, self-training is widely used in both academia and industry by pseudo labeling on readily-available unlabeled data. We propose Debiased, in which the generation and utilization of pseudo labels are decoupled by two independent heads.
  arXiv  Detail & Related papers  (2022-02-15T02:14:33Z)
• Uncertainty-Guided Mutual Consistency Learning for Semi-Supervised Medical Image Segmentation [9.745971699005857]
  We propose a novel uncertainty-guided mutual consistency learning framework for medical image segmentation. It integrates intra-task consistency learning from up-to-date predictions for self-ensembling and cross-task consistency learning from task-level regularization to exploit geometric shape information. Our method achieves performance gains by leveraging unlabeled data and outperforms existing semi-supervised segmentation methods.
  arXiv  Detail & Related papers  (2021-12-05T08:19:41Z)
• Tune it the Right Way: Unsupervised Validation of Domain Adaptation via Soft Neighborhood Density [125.64297244986552]
  We propose an unsupervised validation criterion that measures the density of soft neighborhoods by computing the entropy of the similarity distribution between points. Our criterion is simpler than competing validation methods, yet more effective.
  arXiv  Detail & Related papers  (2021-08-24T17:41:45Z)
• Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning [78.83598532168256]
  Marginal-likelihood based model-selection is rarely used in deep learning due to estimation difficulties. Our work shows that marginal likelihoods can improve generalization and be useful when validation data is unavailable.
  arXiv  Detail & Related papers  (2021-04-11T09:50:24Z)
• Deep Semi-supervised Metric Learning with Dual Alignment for Cervical Cancer Cell Detection [49.78612417406883]
  We propose a novel semi-supervised deep metric learning method for cervical cancer cell detection. Our model learns an embedding metric space and conducts dual alignment of semantic features on both the proposal and prototype levels. We construct a large-scale dataset for semi-supervised cervical cancer cell detection for the first time, consisting of 240,860 cervical cell images.
  arXiv  Detail & Related papers  (2021-04-07T17:11:27Z)
• Matching the Clinical Reality: Accurate OCT-Based Diagnosis From Few Labels [2.891413712995642]
  Unlabeled data is often abundant in the clinic, making machine learning methods based on semi-supervised learning a good match for this setting. Recently proposed MixMatch and FixMatch algorithms have demonstrated promising results in extracting useful representations. We find that both algorithms outperform the transfer learning baseline on all fractions of labelled data.
  arXiv  Detail & Related papers  (2020-10-23T11:47:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.