AlphaMatch: Improving Consistency for Semi-supervised Learning with Alpha-divergence

• URL: http://arxiv.org/abs/2011.11779v1
• Date: Mon, 23 Nov 2020 22:43:45 GMT
• Title: AlphaMatch: Improving Consistency for Semi-supervised Learning with Alpha-divergence
• Authors: Chengyue Gong, Dilin Wang, Qiang Liu
• Abstract summary: Semi-supervised learning (SSL) is a key approach toward more data-efficient machine learning by jointly leverage both labeled and unlabeled data. We propose AlphaMatch, an efficient SSL method that leverages data augmentations, by efficiently enforcing the label consistency between the data points and the augmented data derived from them.
• Score: 44.88886269629515
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Semi-supervised learning (SSL) is a key approach toward more data-efficient machine learning by jointly leverage both labeled and unlabeled data. We propose AlphaMatch, an efficient SSL method that leverages data augmentations, by efficiently enforcing the label consistency between the data points and the augmented data derived from them. Our key technical contribution lies on: 1) using alpha-divergence to prioritize the regularization on data with high confidence, achieving a similar effect as FixMatch but in a more flexible fashion, and 2) proposing an optimization-based, EM-like algorithm to enforce the consistency, which enjoys better convergence than iterative regularization procedures used in recent SSL methods such as FixMatch, UDA, and MixMatch. AlphaMatch is simple and easy to implement, and consistently outperforms prior arts on standard benchmarks, e.g. CIFAR-10, SVHN, CIFAR-100, STL-10. Specifically, we achieve 91.3% test accuracy on CIFAR-10 with just 4 labelled data per class, substantially improving over the previously best 88.7% accuracy achieved by FixMatch.

Related papers

• AstMatch: Adversarial Self-training Consistency Framework for Semi-Supervised Medical Image Segmentation [19.80612796391153]
  Semi-supervised learning (SSL) has shown considerable potential in medical image segmentation. In this work, we propose an adversarial self-training consistency framework (AstMatch) The proposed AstMatch has been extensively evaluated with cutting-edge SSL methods on three public-available datasets.
  arXiv  Detail & Related papers  (2024-06-28T04:38:12Z)
• MaskMatch: Boosting Semi-Supervised Learning Through Mask Autoencoder-Driven Feature Learning [8.255082589733673]
  algo is a novel algorithm that fully utilizes unlabeled data to boost semi-supervised learning. algo integrates a self-supervised learning strategy, i.e., Masked Autoencoder (MAE), that uses all available data to enforce the visual representation learning. algo achieves low error rates of 18.71%, 9.47%, and 3.07%, respectively, on challenging datasets.
  arXiv  Detail & Related papers  (2024-05-10T03:39:54Z)
• JointMatch: A Unified Approach for Diverse and Collaborative Pseudo-Labeling to Semi-Supervised Text Classification [65.268245109828]
  Semi-supervised text classification (SSTC) has gained increasing attention due to its ability to leverage unlabeled data. Existing approaches based on pseudo-labeling suffer from the issues of pseudo-label bias and error accumulation. We propose JointMatch, a holistic approach for SSTC that addresses these challenges by unifying ideas from recent semi-supervised learning.
  arXiv  Detail & Related papers  (2023-10-23T05:43:35Z)
• Class-Aware Contrastive Semi-Supervised Learning [51.205844705156046]
  We propose a general method named Class-aware Contrastive Semi-Supervised Learning (CCSSL) to improve pseudo-label quality and enhance the model's robustness in the real-world setting. Our proposed CCSSL has significant performance improvements over the state-of-the-art SSL methods on the standard datasets CIFAR100 and STL10.
  arXiv  Detail & Related papers  (2022-03-04T12:18:23Z)
• OpenMatch: Open-set Consistency Regularization for Semi-supervised Learning with Outliers [71.08167292329028]
  We propose a novel Open-set Semi-Supervised Learning (OSSL) approach called OpenMatch. OpenMatch unifies FixMatch with novelty detection based on one-vs-all (OVA) classifiers. It achieves state-of-the-art performance on three datasets, and even outperforms a fully supervised model in detecting outliers unseen in unlabeled data on CIFAR10.
  arXiv  Detail & Related papers  (2021-05-28T23:57:15Z)
• CoMatch: Semi-supervised Learning with Contrastive Graph Regularization [86.84486065798735]
  CoMatch is a new semi-supervised learning method that unifies dominant approaches. It achieves state-of-the-art performance on multiple datasets.
  arXiv  Detail & Related papers  (2020-11-23T02:54:57Z)
• Semi-Supervised Learning with Data Augmentation for End-to-End ASR [4.878819328459915]
  We focus on the consistency regularization principle, which has been successfully applied to image classification tasks. We present sequence-to-sequence (seq2seq) versions of the FixMatch and Noisy Student algorithms.
  arXiv  Detail & Related papers  (2020-07-27T21:24:52Z)
• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence [93.91751021370638]
  Semi-supervised learning (SSL) provides an effective means of leveraging unlabeled data to improve a model's performance. In this paper, we demonstrate the power of a simple combination of two common SSL methods: consistency regularization and pseudo-labeling. Our algorithm, FixMatch, first generates pseudo-labels using the model's predictions on weakly-augmented unlabeled images.
  arXiv  Detail & Related papers  (2020-01-21T18:32:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.