Co-learning: Learning from Noisy Labels with Self-supervision
- URL: http://arxiv.org/abs/2108.04063v1
- Date: Thu, 5 Aug 2021 06:20:51 GMT
- Title: Co-learning: Learning from Noisy Labels with Self-supervision
- Authors: Cheng Tan, Jun Xia, Lirong Wu, Stan Z. Li
- Abstract summary: Self-supervised learning works in the absence of labels and thus eliminates the negative impact of noisy labels.
Motivated by co-training with both supervised learning view and self-supervised learning view, we propose a simple yet effective method called Co-learning for learning with noisy labels.
- Score: 28.266156561454327
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Noisy labels, resulting from mistakes in manual labeling or webly data
collecting for supervised learning, can cause neural networks to overfit the
misleading information and degrade the generalization performance.
Self-supervised learning works in the absence of labels and thus eliminates the
negative impact of noisy labels. Motivated by co-training with both supervised
learning view and self-supervised learning view, we propose a simple yet
effective method called Co-learning for learning with noisy labels. Co-learning
performs supervised learning and self-supervised learning in a cooperative way.
The constraints of intrinsic similarity with the self-supervised module and the
structural similarity with the noisily-supervised module are imposed on a
shared common feature encoder to regularize the network to maximize the
agreement between the two constraints. Co-learning is compared with peer
methods on corrupted data from benchmark datasets fairly, and extensive results
are provided which demonstrate that Co-learning is superior to many
state-of-the-art approaches.
Related papers
- Exploiting Minority Pseudo-Labels for Semi-Supervised Semantic Segmentation in Autonomous Driving [2.638145329894673]
We propose a professional training module to enhance minority class learning and a general training module to learn more comprehensive semantic information.
In experiments, our framework demonstrates superior performance compared to state-of-the-art methods on benchmark datasets.
arXiv Detail & Related papers (2024-09-19T11:47:25Z) - A Probabilistic Model Behind Self-Supervised Learning [53.64989127914936]
In self-supervised learning (SSL), representations are learned via an auxiliary task without annotated labels.
We present a generative latent variable model for self-supervised learning.
We show that several families of discriminative SSL, including contrastive methods, induce a comparable distribution over representations.
arXiv Detail & Related papers (2024-02-02T13:31:17Z) - Hyperspherical Consistency Regularization [45.00073340936437]
We explore the relationship between self-supervised learning and supervised learning, and study how self-supervised learning helps robust data-efficient deep learning.
We propose hyperspherical consistency regularization (HCR), a simple yet effective plug-and-play method, to regularize the classifier using feature-dependent information and thus avoid bias from labels.
arXiv Detail & Related papers (2022-06-02T02:41:13Z) - Self-Taught Metric Learning without Labels [47.832107446521626]
We present a novel self-taught framework for unsupervised metric learning.
It alternates between predicting class-equivalence relations between data through a moving average of an embedding model and learning the model with the predicted relations as pseudo labels.
arXiv Detail & Related papers (2022-05-04T05:48:40Z) - CoDiM: Learning with Noisy Labels via Contrastive Semi-Supervised
Learning [58.107679606345165]
Noisy label learning, semi-supervised learning, and contrastive learning are three different strategies for designing learning processes requiring less annotation cost.
We propose CSSL, a unified Contrastive Semi-Supervised Learning algorithm, and CoDiM, a novel algorithm for learning with noisy labels.
arXiv Detail & Related papers (2021-11-23T04:56:40Z) - Robust Mutual Learning for Semi-supervised Semantic Segmentation [23.54885398483688]
We propose robust mutual learning that improves the prior approach in two aspects.
We show that strong data augmentations, model noises and heterogeneous network architectures are essential to alleviate the model coupling.
The proposed robust mutual learning demonstrates state-of-the-art performance on semantic segmentation in low-data regime.
arXiv Detail & Related papers (2021-06-01T16:22:01Z) - ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for
Semi-supervised Continual Learning [52.831894583501395]
Continual learning assumes the incoming data are fully labeled, which might not be applicable in real applications.
We propose deep Online Replay with Discriminator Consistency (ORDisCo) to interdependently learn a classifier with a conditional generative adversarial network (GAN)
We show ORDisCo achieves significant performance improvement on various semi-supervised learning benchmark datasets for SSCL.
arXiv Detail & Related papers (2021-01-02T09:04:14Z) - Can Semantic Labels Assist Self-Supervised Visual Representation
Learning? [194.1681088693248]
We present a new algorithm named Supervised Contrastive Adjustment in Neighborhood (SCAN)
In a series of downstream tasks, SCAN achieves superior performance compared to previous fully-supervised and self-supervised methods.
Our study reveals that semantic labels are useful in assisting self-supervised methods, opening a new direction for the community.
arXiv Detail & Related papers (2020-11-17T13:25:00Z) - Learning to Learn in a Semi-Supervised Fashion [41.38876517851431]
We present a novel meta-learning scheme to address semi-supervised learning from both labeled and unlabeled data.
Our strategy can be viewed as a self-supervised learning scheme, which can be applied to fully supervised learning tasks.
arXiv Detail & Related papers (2020-08-25T17:59:53Z) - Adversarial Self-Supervised Contrastive Learning [62.17538130778111]
Existing adversarial learning approaches mostly use class labels to generate adversarial samples that lead to incorrect predictions.
We propose a novel adversarial attack for unlabeled data, which makes the model confuse the instance-level identities of the perturbed data samples.
We present a self-supervised contrastive learning framework to adversarially train a robust neural network without labeled data.
arXiv Detail & Related papers (2020-06-13T08:24:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.