Semi-Supervised Learning with Multi-Head Co-Training
- URL: http://arxiv.org/abs/2107.04795v1
- Date: Sat, 10 Jul 2021 08:53:14 GMT
- Title: Semi-Supervised Learning with Multi-Head Co-Training
- Authors: Mingcai Chen, Yuntao Du, Yi Zhang, Shuwei Qian, Chongjun Wang
- Abstract summary: We present a simple and efficient co-training algorithm, named Multi-Head Co-Training, for semi-supervised image classification.
Every classification head in the unified model interacts with its peers through a "Weak and Strong Augmentation" strategy.
The effectiveness of Multi-Head Co-Training is demonstrated in an empirical study on standard semi-supervised learning benchmarks.
- Score: 6.675682080298253
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Co-training, extended from self-training, is one of the frameworks for
semi-supervised learning. It works at the cost of training extra classifiers,
where the algorithm should be delicately designed to prevent individual
classifiers from collapsing into each other. In this paper, we present a simple
and efficient co-training algorithm, named Multi-Head Co-Training, for
semi-supervised image classification. By integrating base learners into a
multi-head structure, the model is in a minimal amount of extra parameters.
Every classification head in the unified model interacts with its peers through
a "Weak and Strong Augmentation" strategy, achieving single-view co-training
without promoting diversity explicitly. The effectiveness of Multi-Head
Co-Training is demonstrated in an empirical study on standard semi-supervised
learning benchmarks.
Related papers
- Semi-supervised learning made simple with self-supervised clustering [65.98152950607707]
Self-supervised learning models have been shown to learn rich visual representations without requiring human annotations.
We propose a conceptually simple yet empirically powerful approach to turn clustering-based self-supervised methods into semi-supervised learners.
arXiv Detail & Related papers (2023-06-13T01:09:18Z) - FedCBO: Reaching Group Consensus in Clustered Federated Learning through
Consensus-based Optimization [1.911678487931003]
Federated learning seeks to integrate the training learning models from multiple users, each user having their own data set, in a way that is sensitive to data privacy and to communication loss constraints.
In this paper, we propose a novel solution to a global, clustered problem of federated learning that is inspired by ideas in consensus-based optimization (CBO)
Our new CBO-type method is based on a system of interacting particles that is oblivious to group.
arXiv Detail & Related papers (2023-05-04T15:02:09Z) - Few-shot Classification via Ensemble Learning with Multi-Order
Statistics [9.145742362513932]
We show that leveraging ensemble learning on the base classes can correspondingly reduce the true error in the novel classes.
A novel method named Ensemble Learning with Multi-Order Statistics (ELMOS) is proposed in this paper.
We show that our method can produce a state-of-the-art performance on multiple few-shot classification benchmark datasets.
arXiv Detail & Related papers (2023-04-30T11:41:01Z) - Deep Negative Correlation Classification [82.45045814842595]
Existing deep ensemble methods naively train many different models and then aggregate their predictions.
We propose deep negative correlation classification (DNCC)
DNCC yields a deep classification ensemble where the individual estimator is both accurate and negatively correlated.
arXiv Detail & Related papers (2022-12-14T07:35:20Z) - Towards All-in-one Pre-training via Maximizing Multi-modal Mutual
Information [77.80071279597665]
We propose an all-in-one single-stage pre-training approach, named Maximizing Multi-modal Mutual Information Pre-training (M3I Pre-training)
Our approach achieves better performance than previous pre-training methods on various vision benchmarks, including ImageNet classification, object detection, LVIS long-tailed object detection, and ADE20k semantic segmentation.
arXiv Detail & Related papers (2022-11-17T18:59:49Z) - Efficient Self-Ensemble Framework for Semantic Segmentation [1.0819401241801994]
We propose to leverage the performance boost offered by ensemble methods to enhance semantic segmentation.
Our self-ensemble framework takes advantage of the multi-scale features set produced by feature pyramid network methods.
Our model can be trained end-to-end, alleviating the traditional cumbersome multi-stage training of ensembles.
arXiv Detail & Related papers (2021-11-26T00:35:09Z) - Partner-Assisted Learning for Few-Shot Image Classification [54.66864961784989]
Few-shot Learning has been studied to mimic human visual capabilities and learn effective models without the need of exhaustive human annotation.
In this paper, we focus on the design of training strategy to obtain an elemental representation such that the prototype of each novel class can be estimated from a few labeled samples.
We propose a two-stage training scheme, which first trains a partner encoder to model pair-wise similarities and extract features serving as soft-anchors, and then trains a main encoder by aligning its outputs with soft-anchors while attempting to maximize classification performance.
arXiv Detail & Related papers (2021-09-15T22:46:19Z) - Semi-Supervised Few-Shot Classification with Deep Invertible Hybrid
Models [4.189643331553922]
We propose a deep invertible hybrid model which integrates discriminative and generative learning at a latent space level for semi-supervised few-shot classification.
Our main originality lies in our integration of these components at a latent space level, which is effective in preventing overfitting.
arXiv Detail & Related papers (2021-05-22T05:55:16Z) - Combining Deep Generative Models and Multi-lingual Pretraining for
Semi-supervised Document Classification [49.47925519332164]
We combine semi-supervised deep generative models and multi-lingual pretraining to form a pipeline for document classification task.
Our framework is highly competitive and outperforms the state-of-the-art counterparts in low-resource settings across several languages.
arXiv Detail & Related papers (2021-01-26T11:26:14Z) - Unpaired Multi-modal Segmentation via Knowledge Distillation [77.39798870702174]
We propose a novel learning scheme for unpaired cross-modality image segmentation.
In our method, we heavily reuse network parameters, by sharing all convolutional kernels across CT and MRI.
We have extensively validated our approach on two multi-class segmentation problems.
arXiv Detail & Related papers (2020-01-06T20:03:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.