Boosting Consistency in Dual Training for Long-Tailed Semi-Supervised Learning

• URL: http://arxiv.org/abs/2406.13187v1
• Date: Wed, 19 Jun 2024 03:35:26 GMT
• Title: Boosting Consistency in Dual Training for Long-Tailed Semi-Supervised Learning
• Authors: Kai Gan, Tong Wei, Min-Ling Zhang,
• Abstract summary: Long-tailed semi-supervised learning (LTSSL) algorithms assume that the class distributions of labeled and unlabeled data are almost identical. We propose a new simple method that can effectively utilize unlabeled data from unknown class distributions. We show that BOAT achieves state-of-the-art performance on a variety of standard LTSSL benchmarks.
• Score: 49.07038093130949
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While long-tailed semi-supervised learning (LTSSL) has received tremendous attention in many real-world classification problems, existing LTSSL algorithms typically assume that the class distributions of labeled and unlabeled data are almost identical. Those LTSSL algorithms built upon the assumption can severely suffer when the class distributions of labeled and unlabeled data are mismatched since they utilize biased pseudo-labels from the model. To alleviate this problem, we propose a new simple method that can effectively utilize unlabeled data from unknown class distributions through Boosting cOnsistency in duAl Training (BOAT). Specifically, we construct the standard and balanced branch to ensure the performance of the head and tail classes, respectively. Throughout the training process, the two branches incrementally converge and interact with each other, eventually resulting in commendable performance across all classes. Despite its simplicity, we show that BOAT achieves state-of-the-art performance on a variety of standard LTSSL benchmarks, e.g., an averaged 2.7% absolute increase in test accuracy against existing algorithms when the class distributions of labeled and unlabeled data are mismatched. Even when the class distributions are identical, BOAT consistently outperforms many sophisticated LTSSL algorithms. We carry out extensive ablation studies to tease apart the factors that are the most important to the success of BOAT. The source code is available at https://github.com/Gank0078/BOAT.

Related papers

• Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition [50.61991746981703]
  Current state-of-the-art LTSSL approaches rely on high-quality pseudo-labels for large-scale unlabeled data. This paper introduces a novel probabilistic framework that unifies various recent proposals in long-tail learning. We introduce a continuous contrastive learning method, CCL, extending our framework to unlabeled data using reliable and smoothed pseudo-labels.
  arXiv  Detail & Related papers  (2024-10-08T15:06:10Z)
• Three Heads Are Better Than One: Complementary Experts for Long-Tailed Semi-supervised Learning [74.44500692632778]
  We propose a novel method named ComPlementary Experts (CPE) to model various class distributions. CPE achieves state-of-the-art performances on CIFAR-10-LT, CIFAR-100-LT, and STL-10-LT dataset benchmarks.
  arXiv  Detail & Related papers  (2023-12-25T11:54:07Z)
• JointMatch: A Unified Approach for Diverse and Collaborative Pseudo-Labeling to Semi-Supervised Text Classification [65.268245109828]
  Semi-supervised text classification (SSTC) has gained increasing attention due to its ability to leverage unlabeled data. Existing approaches based on pseudo-labeling suffer from the issues of pseudo-label bias and error accumulation. We propose JointMatch, a holistic approach for SSTC that addresses these challenges by unifying ideas from recent semi-supervised learning.
  arXiv  Detail & Related papers  (2023-10-23T05:43:35Z)
• Flexible Distribution Alignment: Towards Long-tailed Semi-supervised Learning with Proper Calibration [18.376601653387315]
  Longtailed semi-supervised learning (LTSSL) represents a practical scenario for semi-supervised applications. This problem is often aggravated by discrepancies between labeled and unlabeled class distributions. We introduce Flexible Distribution Alignment (FlexDA), a novel adaptive logit-adjusted loss framework.
  arXiv  Detail & Related papers  (2023-06-07T17:50:59Z)
• On Non-Random Missing Labels in Semi-Supervised Learning [114.62655062520425]
  Semi-Supervised Learning (SSL) is fundamentally a missing label problem. We explicitly incorporate "class" into SSL. Our method not only significantly outperforms existing baselines but also surpasses other label bias removal SSL methods.
  arXiv  Detail & Related papers  (2022-06-29T22:01:29Z)
• ABC: Auxiliary Balanced Classifier for Class-imbalanced Semi-supervised Learning [6.866717993664787]
  Existing semi-supervised learning (SSL) algorithms assume class-balanced datasets. We propose a scalable class-imbalanced SSL algorithm that can effectively use unlabeled data. The proposed algorithm achieves state-of-the-art performance in various class-imbalanced SSL experiments using four benchmark datasets.
  arXiv  Detail & Related papers  (2021-10-20T04:07:48Z)
• BiSTF: Bilateral-Branch Self-Training Framework for Semi-Supervised Large-scale Fine-Grained Recognition [28.06659482245647]
  Semi-supervised Fine-Grained Recognition is a challenge task due to data imbalance, high interclass similarity and domain mismatch. We propose Bilateral-Branch Self-Training Framework (BiSTF) to improve existing semi-balanced and domain-shifted fine-grained data. We show BiSTF outperforms the existing state-of-the-art SSL on Semi-iNat dataset.
  arXiv  Detail & Related papers  (2021-07-14T15:28:54Z)
• Boosting the Performance of Semi-Supervised Learning with Unsupervised Clustering [10.033658645311188]
  We show that ignoring labels altogether for whole epochs intermittently during training can significantly improve performance in the small sample regime. We demonstrate our method's efficacy in boosting several state-of-the-art SSL algorithms.
  arXiv  Detail & Related papers  (2020-12-01T14:19:14Z)
• Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning [126.31716228319902]
  We develop Distribution Aligning Refinery of Pseudo-label (DARP) algorithm. We show that DARP is provably and efficiently compatible with state-of-the-art SSL schemes.
  arXiv  Detail & Related papers  (2020-07-17T09:16:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.