RegMixMatch: Optimizing Mixup Utilization in Semi-Supervised Learning

• URL: http://arxiv.org/abs/2412.10741v1
• Date: Sat, 14 Dec 2024 08:22:49 GMT
• Title: RegMixMatch: Optimizing Mixup Utilization in Semi-Supervised Learning
• Authors: Haorong Han, Jidong Yuan, Chixuan Wei, Zhongyang Yu,
• Abstract summary: Semi-supervised RegMixup achieves state-of-the-art performance across various SSL benchmarks. We develop a class-aware Mixup technique that integrates information from the top-2 predicted classes into low-confidence samples and their artificial labels.
• Score: 0.7274730603514221
• License:
• Abstract: Consistency regularization and pseudo-labeling have significantly advanced semi-supervised learning (SSL). Prior works have effectively employed Mixup for consistency regularization in SSL. However, our findings indicate that applying Mixup for consistency regularization may degrade SSL performance by compromising the purity of artificial labels. Moreover, most pseudo-labeling based methods utilize thresholding strategy to exclude low-confidence data, aiming to mitigate confirmation bias; however, this approach limits the utility of unlabeled samples. To address these challenges, we propose RegMixMatch, a novel framework that optimizes the use of Mixup with both high- and low-confidence samples in SSL. First, we introduce semi-supervised RegMixup, which effectively addresses reduced artificial labels purity by using both mixed samples and clean samples for training. Second, we develop a class-aware Mixup technique that integrates information from the top-2 predicted classes into low-confidence samples and their artificial labels, reducing the confirmation bias associated with these samples and enhancing their effective utilization. Experimental results demonstrate that RegMixMatch achieves state-of-the-art performance across various SSL benchmarks.

Related papers

• A Channel-ensemble Approach: Unbiased and Low-variance Pseudo-labels is Critical for Semi-supervised Classification [61.473485511491795]
  Semi-supervised learning (SSL) is a practical challenge in computer vision. Pseudo-label (PL) methods, e.g., FixMatch and FreeMatch, obtain the State Of The Art (SOTA) performances in SSL. We propose a lightweight channel-based ensemble method to consolidate multiple inferior PLs into the theoretically guaranteed unbiased and low-variance one.
  arXiv  Detail & Related papers  (2024-03-27T09:49:37Z)
• RankMatch: A Novel Approach to Semi-Supervised Label Distribution Learning Leveraging Inter-label Correlations [52.549807652527306]
  This paper introduces RankMatch, an innovative approach for Semi-Supervised Label Distribution Learning (SSLDL) RankMatch effectively utilizes a small number of labeled examples in conjunction with a larger quantity of unlabeled data. We establish a theoretical generalization bound for RankMatch, and through extensive experiments, demonstrate its superiority in performance against existing SSLDL methods.
  arXiv  Detail & Related papers  (2023-12-11T12:47:29Z)
• Learning with Noisy Labels Using Collaborative Sample Selection and Contrastive Semi-Supervised Learning [76.00798972439004]
  Collaborative Sample Selection (CSS) removes noisy samples from identified clean set. We introduce a co-training mechanism with a contrastive loss in semi-supervised learning.
  arXiv  Detail & Related papers  (2023-10-24T05:37:20Z)
• Shrinking Class Space for Enhanced Certainty in Semi-Supervised Learning [59.44422468242455]
  We propose a novel method dubbed ShrinkMatch to learn uncertain samples. For each uncertain sample, it adaptively seeks a shrunk class space, which merely contains the original top-1 class. We then impose a consistency regularization between a pair of strongly and weakly augmented samples in the shrunk space to strive for discriminative representations.
  arXiv  Detail & Related papers  (2023-08-13T14:05:24Z)
• InPL: Pseudo-labeling the Inliers First for Imbalanced Semi-supervised Learning [34.062061310242385]
  We present a new perspective of pseudo-labeling for imbalanced semi-supervised learning (SSL) We measure whether an unlabeled sample is likely to be in-distribution'' or out-of-distribution'' Experiments demonstrate that our energy-based pseudo-labeling method, textbfInPL, significantly outperforms confidence-based methods on imbalanced SSL benchmarks.
  arXiv  Detail & Related papers  (2023-03-13T16:45:41Z)
• Harnessing Hard Mixed Samples with Decoupled Regularizer [69.98746081734441]
  Mixup is an efficient data augmentation approach that improves the generalization of neural networks by smoothing the decision boundary with mixed data. In this paper, we propose an efficient mixup objective function with a decoupled regularizer named Decoupled Mixup (DM) DM can adaptively utilize hard mixed samples to mine discriminative features without losing the original smoothness of mixup.
  arXiv  Detail & Related papers  (2022-03-21T07:12:18Z)
• Boosting Discriminative Visual Representation Learning with Scenario-Agnostic Mixup [54.09898347820941]
  We propose textbfScenario-textbfAgnostic textbfMixup (SAMix) for both Self-supervised Learning (SSL) and supervised learning (SL) scenarios. Specifically, we hypothesize and verify the objective function of mixup generation as optimizing local smoothness between two mixed classes. A label-free generation sub-network is designed, which effectively provides non-trivial mixup samples and improves transferable abilities.
  arXiv  Detail & Related papers  (2021-11-30T14:49:59Z)
• SMILE: Self-Distilled MIxup for Efficient Transfer LEarning [42.59451803498095]
  In this work, we propose SMILE - Self-Distilled Mixup for EffIcient Transfer LEarning. With mixed images as inputs, SMILE regularizes the outputs of CNN feature extractors to learn from the mixed feature vectors of inputs. The triple regularizer balances the mixup effects in both feature and label spaces while bounding the linearity in-between samples for pre-training tasks.
  arXiv  Detail & Related papers  (2021-03-25T16:02:21Z)
• Consistency Regularization with Generative Adversarial Networks for Semi-Supervised Learning [2.9707483702447783]
  Adversarial Adversarial Networks (GANs) based semi-supervised learning (SSL) approaches are shown to improve classification performance by utilizing a large number of unlabeled samples. However, their performance still lags behind the state-of-the-art non-GAN based SSL approaches. We identify that the main reason for this is the lack of consistency in class probability predictions on the same image under local perturbations.
  arXiv  Detail & Related papers  (2020-07-08T01:47:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.