Unsupervised Representation Learning Meets Pseudo-Label Supervised Self-Distillation: A New Approach to Rare Disease Classification

• URL: http://arxiv.org/abs/2110.04558v1
• Date: Sat, 9 Oct 2021 12:56:09 GMT
• Title: Unsupervised Representation Learning Meets Pseudo-Label Supervised Self-Distillation: A New Approach to Rare Disease Classification
• Authors: Jinghan Sun, Dong Wei, Kai Ma, Liansheng Wang, and Yefeng Zheng
• Abstract summary: We propose a novel hybrid approach to rare disease classification, featuring two key novelties. First, we adopt the unsupervised representation learning (URL) based on self-supervising contrastive loss. Second, we integrate the URL with pseudo-label supervised classification for effective self-distillation of the knowledge about the rare diseases.
• Score: 26.864435224276964
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Rare diseases are characterized by low prevalence and are often chronically debilitating or life-threatening. Imaging-based classification of rare diseases is challenging due to the severe shortage in training examples. Few-shot learning (FSL) methods tackle this challenge by extracting generalizable prior knowledge from a large base dataset of common diseases and normal controls, and transferring the knowledge to rare diseases. Yet, most existing methods require the base dataset to be labeled and do not make full use of the precious examples of the rare diseases. To this end, we propose in this work a novel hybrid approach to rare disease classification, featuring two key novelties targeted at the above drawbacks. First, we adopt the unsupervised representation learning (URL) based on self-supervising contrastive loss, whereby to eliminate the overhead in labeling the base dataset. Second, we integrate the URL with pseudo-label supervised classification for effective self-distillation of the knowledge about the rare diseases, composing a hybrid approach taking advantages of both unsupervised and (pseudo-) supervised learning on the base dataset. Experimental results on classification of rare skin lesions show that our hybrid approach substantially outperforms existing FSL methods (including those using fully supervised base dataset) for rare disease classification via effective integration of the URL and pseudo-label driven self-distillation, thus establishing a new state of the art.

Related papers

• A Noisy-Label-Learning Formulation for Immune Repertoire Classification and Disease-Associated Immune Receptor Sequence Identification [7.619591696318021]
  Immune repertoire classification is a frontier research topic in computational biology. We propose a noisy-label-learning formulation to solve the immune repertoire classification task. Experiments on the Cytomegalovirus (CMV) and Cancer datasets demonstrate our method's effectiveness and superior performance.
  arXiv  Detail & Related papers  (2023-07-29T09:19:27Z)
• Leveraging Unlabelled Data in Multiple-Instance Learning Problems for Improved Detection of Parkinsonian Tremor in Free-Living Conditions [80.88681952022479]
  We introduce a new method for combining semi-supervised with multiple-instance learning. We show that by leveraging the unlabelled data of 454 subjects we can achieve large performance gains in per-subject tremor detection.
  arXiv  Detail & Related papers  (2023-04-29T12:25:10Z)
• Semi-Supervised Relational Contrastive Learning [8.5285439285139]
  We present a novel semi-supervised learning model that leverages self-supervised contrastive loss and consistency. We validate against the ISIC 2018 Challenge benchmark skin lesion classification and demonstrate the effectiveness of our method on varying amounts of labeled data.
  arXiv  Detail & Related papers  (2023-04-11T08:14:30Z)
• Hierarchical Semi-Supervised Contrastive Learning for Contamination-Resistant Anomaly Detection [81.07346419422605]
  Anomaly detection aims at identifying deviant samples from the normal data distribution. Contrastive learning has provided a successful way to sample representation that enables effective discrimination on anomalies. We propose a novel hierarchical semi-supervised contrastive learning framework, for contamination-resistant anomaly detection.
  arXiv  Detail & Related papers  (2022-07-24T18:49:26Z)
• Learning Discriminative Representation via Metric Learning for Imbalanced Medical Image Classification [52.94051907952536]
  We propose embedding metric learning into the first stage of the two-stage framework specially to help the feature extractor learn to extract more discriminative feature representations. Experiments mainly on three medical image datasets show that the proposed approach consistently outperforms existing onestage and two-stage approaches.
  arXiv  Detail & Related papers  (2022-07-14T14:57:01Z)
• LifeLonger: A Benchmark for Continual Disease Classification [59.13735398630546]
  We introduce LifeLonger, a benchmark for continual disease classification on the MedMNIST collection. Task and class incremental learning of diseases address the issue of classifying new samples without re-training the models from scratch. Cross-domain incremental learning addresses the issue of dealing with datasets originating from different institutions while retaining the previously obtained knowledge.
  arXiv  Detail & Related papers  (2022-04-12T12:25:05Z)
• Hierarchical Knowledge Guided Learning for Real-world Retinal Diseases Recognition [20.88407972858568]
  Some recently published datasets in ophthalmology AI consist of more than 40 kinds of retinal diseases with complex abnormalities and variable morbidity. From a modeling perspective, most deep learning models trained on these datasets may lack the ability to generalize to rare diseases. This paper presents a novel method that enables the deep neural network to learn from a long-tailed fundus database for various retinal disease recognition.
  arXiv  Detail & Related papers  (2021-11-17T05:44:39Z)
• Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype Prediction [55.94378672172967]
  We focus on few-shot disease subtype prediction problem, identifying subgroups of similar patients. We introduce meta learning techniques to develop a new model, which can extract the common experience or knowledge from interrelated clinical tasks. Our new model is built upon a carefully designed meta-learner, called Prototypical Network, that is a simple yet effective meta learning machine for few-shot image classification.
  arXiv  Detail & Related papers  (2020-09-02T02:50:30Z)
• Semi-supervised Medical Image Classification with Relation-driven Self-ensembling Model [71.80319052891817]
  We present a relation-driven semi-supervised framework for medical image classification. It exploits the unlabeled data by encouraging the prediction consistency of given input under perturbations. Our method outperforms many state-of-the-art semi-supervised learning methods on both single-label and multi-label image classification scenarios.
  arXiv  Detail & Related papers  (2020-05-15T06:57:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.