MaskMatch: Boosting Semi-Supervised Learning Through Mask Autoencoder-Driven Feature Learning

• URL: http://arxiv.org/abs/2405.06227v1
• Date: Fri, 10 May 2024 03:39:54 GMT
• Title: MaskMatch: Boosting Semi-Supervised Learning Through Mask Autoencoder-Driven Feature Learning
• Authors: Wenjin Zhang, Keyi Li, Sen Yang, Chenyang Gao, Wanzhao Yang, Sifan Yuan, Ivan Marsic,
• Abstract summary: algo is a novel algorithm that fully utilizes unlabeled data to boost semi-supervised learning. algo integrates a self-supervised learning strategy, i.e., Masked Autoencoder (MAE), that uses all available data to enforce the visual representation learning. algo achieves low error rates of 18.71%, 9.47%, and 3.07%, respectively, on challenging datasets.
• Score: 8.255082589733673
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conventional methods in semi-supervised learning (SSL) often face challenges related to limited data utilization, mainly due to their reliance on threshold-based techniques for selecting high-confidence unlabeled data during training. Various efforts (e.g., FreeMatch) have been made to enhance data utilization by tweaking the thresholds, yet none have managed to use 100% of the available data. To overcome this limitation and improve SSL performance, we introduce \algo, a novel algorithm that fully utilizes unlabeled data to boost semi-supervised learning. \algo integrates a self-supervised learning strategy, i.e., Masked Autoencoder (MAE), that uses all available data to enforce the visual representation learning. This enables the SSL algorithm to leverage all available data, including samples typically filtered out by traditional methods. In addition, we propose a synthetic data training approach to further increase data utilization and improve generalization. These innovations lead \algo to achieve state-of-the-art results on challenging datasets. For instance, on CIFAR-100 with 2 labels per class, STL-10 with 4 labels per class, and Euro-SAT with 2 labels per class, \algo achieves low error rates of 18.71%, 9.47%, and 3.07%, respectively. The code will be made publicly available.

Related papers

• Improving 3D Semi-supervised Learning by Effectively Utilizing All Unlabelled Data [21.015491263215786]
  Semi-supervised learning (SSL) has shown its effectiveness in learning effective 3D representation from a small amount of labelled data. We propose AllMatch, a novel SSL-based 3D classification framework that effectively utilizes all the unlabelled samples.
  arXiv  Detail & Related papers  (2024-09-21T01:53:52Z)
• FlatMatch: Bridging Labeled Data and Unlabeled Data with Cross-Sharpness for Semi-Supervised Learning [73.13448439554497]
  Semi-Supervised Learning (SSL) has been an effective way to leverage abundant unlabeled data with extremely scarce labeled data. Most SSL methods are commonly based on instance-wise consistency between different data transformations. We propose FlatMatch which minimizes a cross-sharpness measure to ensure consistent learning performance between the two datasets.
  arXiv  Detail & Related papers  (2023-10-25T06:57:59Z)
• Boosting Semi-Supervised Learning by Exploiting All Unlabeled Data [21.6350640726058]
  Semi-supervised learning (SSL) has attracted enormous attention due to its vast potential of mitigating the dependence on large labeled datasets. We propose two novel techniques: Entropy Meaning Loss (EML) and Adaptive Negative Learning (ANL) We integrate these techniques with FixMatch, and develop a simple yet powerful framework called FullMatch.
  arXiv  Detail & Related papers  (2023-03-20T12:44:11Z)
• Semantic Segmentation with Active Semi-Supervised Representation Learning [23.79742108127707]
  We train an effective semantic segmentation algorithm with significantly lesser labeled data. We extend the prior state-of-the-art S4AL algorithm by replacing its mean teacher approach for semi-supervised learning with a self-training approach. We evaluate our method on CamVid and CityScapes datasets, the de-facto standards for active learning for semantic segmentation.
  arXiv  Detail & Related papers  (2022-10-16T00:21:43Z)
• Is margin all you need? An extensive empirical study of active learning on tabular data [66.18464006872345]
  We analyze the performance of a variety of active learning algorithms on 69 real-world datasets from the OpenML-CC18 benchmark. Surprisingly, we find that the classical margin sampling technique matches or outperforms all others, including current state-of-art.
  arXiv  Detail & Related papers  (2022-10-07T21:18:24Z)
• Towards Realistic Semi-Supervised Learning [73.59557447798134]
  We propose a novel approach to tackle SSL in open-world setting, where we simultaneously learn to classify known and unknown classes. Our approach substantially outperforms the existing state-of-the-art on seven diverse datasets.
  arXiv  Detail & Related papers  (2022-07-05T19:04:43Z)
• Collaborative Intelligence Orchestration: Inconsistency-Based Fusion of Semi-Supervised Learning and Active Learning [60.26659373318915]
  Active learning (AL) and semi-supervised learning (SSL) are two effective, but often isolated, means to alleviate the data-hungry problem. We propose an innovative Inconsistency-based virtual aDvErial algorithm to further investigate SSL-AL's potential superiority. Two real-world case studies visualize the practical industrial value of applying and deploying the proposed data sampling algorithm.
  arXiv  Detail & Related papers  (2022-06-07T13:28:43Z)
• Class-Aware Contrastive Semi-Supervised Learning [51.205844705156046]
  We propose a general method named Class-aware Contrastive Semi-Supervised Learning (CCSSL) to improve pseudo-label quality and enhance the model's robustness in the real-world setting. Our proposed CCSSL has significant performance improvements over the state-of-the-art SSL methods on the standard datasets CIFAR100 and STL10.
  arXiv  Detail & Related papers  (2022-03-04T12:18:23Z)
• SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption [72.35532598131176]
  We propose SCARF, a technique for contrastive learning, where views are formed by corrupting a random subset of features. We show that SCARF complements existing strategies and outperforms alternatives like autoencoders.
  arXiv  Detail & Related papers  (2021-06-29T08:08:33Z)
• Relieving the Plateau: Active Semi-Supervised Learning for a Better Landscape [2.3046646540823916]
  Semi-supervised learning (SSL) leverages unlabeled data that are more accessible than their labeled counterparts. Active learning (AL) selects unlabeled instances to be annotated by a human-in-the-loop in hopes of better performance with less labeled data. We propose convergence rate control (CRC), an AL algorithm that selects unlabeled data to improve the problem conditioning upon inclusion to the labeled set.
  arXiv  Detail & Related papers  (2021-04-08T06:03:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.