Semi-Supervised Learning with Pseudo-Negative Labels for Image Classification

• URL: http://arxiv.org/abs/2301.03976v1
• Date: Tue, 10 Jan 2023 14:15:17 GMT
• Title: Semi-Supervised Learning with Pseudo-Negative Labels for Image Classification
• Authors: Hao Xu, Hui Xiao, Huazheng Hao, Li Dong, Xiaojie Qiu and Chengbin Peng
• Abstract summary: We propose a mutual learning framework based on pseudo-negative labels. By reducing the prediction probability on pseudo-negative labels, the dual model can improve its prediction ability. Our framework achieves state-of-the-art results on several main benchmarks.
• Score: 14.100569951592417
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semi-supervised learning frameworks usually adopt mutual learning approaches with multiple submodels to learn from different perspectives. To avoid transferring erroneous pseudo labels between these submodels, a high threshold is usually used to filter out a large number of low-confidence predictions for unlabeled data. However, such filtering can not fully exploit unlabeled data with low prediction confidence. To overcome this problem, in this work, we propose a mutual learning framework based on pseudo-negative labels. Negative labels are those that a corresponding data item does not belong. In each iteration, one submodel generates pseudo-negative labels for each data item, and the other submodel learns from these labels. The role of the two submodels exchanges after each iteration until convergence. By reducing the prediction probability on pseudo-negative labels, the dual model can improve its prediction ability. We also propose a mechanism to select a few pseudo-negative labels to feed into submodels. In the experiments, our framework achieves state-of-the-art results on several main benchmarks. Specifically, with our framework, the error rates of the 13-layer CNN model are 9.35% and 7.94% for CIFAR-10 with 1000 and 4000 labels, respectively. In addition, for the non-augmented MNIST with only 20 labels, the error rate is 0.81% by our framework, which is much smaller than that of other approaches. Our approach also demonstrates a significant performance improvement in domain adaptation.

Related papers

• Generating Unbiased Pseudo-labels via a Theoretically Guaranteed Chebyshev Constraint to Unify Semi-supervised Classification and Regression [57.17120203327993]
  threshold-to-pseudo label process (T2L) in classification uses confidence to determine the quality of label. In nature, regression also requires unbiased methods to generate high-quality labels. We propose a theoretically guaranteed constraint for generating unbiased labels based on Chebyshev's inequality.
  arXiv  Detail & Related papers  (2023-11-03T08:39:35Z)
• Label-Retrieval-Augmented Diffusion Models for Learning from Noisy Labels [61.97359362447732]
  Learning from noisy labels is an important and long-standing problem in machine learning for real applications. In this paper, we reformulate the label-noise problem from a generative-model perspective. Our model achieves new state-of-the-art (SOTA) results on all the standard real-world benchmark datasets.
  arXiv  Detail & Related papers  (2023-05-31T03:01:36Z)
• Bridging the Gap between Model Explanations in Partially Annotated Multi-label Classification [85.76130799062379]
  We study how false negative labels affect the model's explanation. We propose to boost the attribution scores of the model trained with partial labels to make its explanation resemble that of the model trained with full labels.
  arXiv  Detail & Related papers  (2023-04-04T14:00:59Z)
• Dist-PU: Positive-Unlabeled Learning from a Label Distribution Perspective [89.5370481649529]
  We propose a label distribution perspective for PU learning in this paper. Motivated by this, we propose to pursue the label distribution consistency between predicted and ground-truth label distributions. Experiments on three benchmark datasets validate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-12-06T07:38:29Z)
• Improving Zero-Shot Models with Label Distribution Priors [33.51714665243138]
  We propose a new approach, CLIPPR, which adapts zero-shot models for regression and classification on unlabelled datasets. We demonstrate an improvement of 28% in mean absolute error on the UTK age regression task. We also present promising results for classification benchmarks, improving the classification accuracy on the ImageNet dataset by 2.83%, without using any labels.
  arXiv  Detail & Related papers  (2022-12-01T18:59:03Z)
• Conformal Predictor for Improving Zero-shot Text Classification Efficiency [37.745518881553416]
  We reduce the average inference time for NLI- and NSP-based models by 25.6% and 22.2% respectively. With a suitable CP for each dataset, we reduce the average inference time for NLI- and NSP-based models by 25.6% and 22.2% respectively.
  arXiv  Detail & Related papers  (2022-10-23T05:19:50Z)
• Pseudo-Label Noise Suppression Techniques for Semi-Supervised Semantic Segmentation [21.163070161951868]
  Semi-consuming learning (SSL) can reduce the need for large labelled datasets by incorporating unsupervised data into the training. Current SSL approaches use an initially supervised trained model to generate predictions for unlabelled images, called pseudo-labels. We use three mechanisms to control pseudo-label noise and errors.
  arXiv  Detail & Related papers  (2022-10-19T09:46:27Z)
• FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning [46.95063831057502]
  We propose emphFreeMatch to define and adjust the confidence threshold in a self-adaptive manner according to the model's learning status. FreeMatch achieves textbf5.78%, textbf13.59%, and textbf1.28% error rate reduction over the latest state-of-the-art method FlexMatch on CIFAR-10 with 1 label per class.
  arXiv  Detail & Related papers  (2022-05-15T10:07:52Z)
• Cross-Model Pseudo-Labeling for Semi-Supervised Action Recognition [98.25592165484737]
  We propose a more effective pseudo-labeling scheme, called Cross-Model Pseudo-Labeling (CMPL) CMPL achieves $17.6%$ and $25.1%$ Top-1 accuracy on Kinetics-400 and UCF-101 using only the RGB modality and $1%$ labeled data, respectively.
  arXiv  Detail & Related papers  (2021-12-17T18:59:41Z)
• Multi-class Probabilistic Bounds for Self-learning [13.875239300089861]
  Pseudo-labeling is prone to error and runs the risk of adding noisy labels into unlabeled training data. We present a probabilistic framework for analyzing self-learning in the multi-class classification scenario with partially labeled data.
  arXiv  Detail & Related papers  (2021-09-29T13:57:37Z)
• A Theory-Driven Self-Labeling Refinement Method for Contrastive Representation Learning [111.05365744744437]
  Unsupervised contrastive learning labels crops of the same image as positives, and other image crops as negatives. In this work, we first prove that for contrastive learning, inaccurate label assignment heavily impairs its generalization for semantic instance discrimination. Inspired by this theory, we propose a novel self-labeling refinement approach for contrastive learning.
  arXiv  Detail & Related papers  (2021-06-28T14:24:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.