Twin Contrastive Learning with Noisy Labels
- URL: http://arxiv.org/abs/2303.06930v1
- Date: Mon, 13 Mar 2023 08:53:47 GMT
- Title: Twin Contrastive Learning with Noisy Labels
- Authors: Zhizhong Huang and Junping Zhang and Hongming Shan
- Abstract summary: We present TCL, a novel twin contrastive learning model to learn robust representations and handle noisy labels for classification.
TCL achieves 7.5% improvements on CIFAR-10 with 90% noisy label -- an extremely noisy scenario.
- Score: 45.31997043789471
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Learning from noisy data is a challenging task that significantly degenerates
the model performance. In this paper, we present TCL, a novel twin contrastive
learning model to learn robust representations and handle noisy labels for
classification. Specifically, we construct a Gaussian mixture model (GMM) over
the representations by injecting the supervised model predictions into GMM to
link label-free latent variables in GMM with label-noisy annotations. Then, TCL
detects the examples with wrong labels as the out-of-distribution examples by
another two-component GMM, taking into account the data distribution. We
further propose a cross-supervision with an entropy regularization loss that
bootstraps the true targets from model predictions to handle the noisy labels.
As a result, TCL can learn discriminative representations aligned with
estimated labels through mixup and contrastive learning. Extensive experimental
results on several standard benchmarks and real-world datasets demonstrate the
superior performance of TCL. In particular, TCL achieves 7.5\% improvements on
CIFAR-10 with 90\% noisy label -- an extremely noisy scenario. The source code
is available at \url{https://github.com/Hzzone/TCL}.
Related papers
- Exploring Beyond Logits: Hierarchical Dynamic Labeling Based on Embeddings for Semi-Supervised Classification [49.09505771145326]
We propose a Hierarchical Dynamic Labeling (HDL) algorithm that does not depend on model predictions and utilizes image embeddings to generate sample labels.
Our approach has the potential to change the paradigm of pseudo-label generation in semi-supervised learning.
arXiv Detail & Related papers (2024-04-26T06:00:27Z) - Label Denoising through Cross-Model Agreement [43.5145547124009]
Memorizing noisy labels could affect the learning of the model, leading to sub-optimal performances.
We propose a novel framework to learn robust machine-learning models from noisy labels.
arXiv Detail & Related papers (2023-08-27T00:31:04Z) - Channel-Wise Contrastive Learning for Learning with Noisy Labels [60.46434734808148]
We introduce channel-wise contrastive learning (CWCL) to distinguish authentic label information from noise.
Unlike conventional instance-wise contrastive learning (IWCL), CWCL tends to yield more nuanced and resilient features aligned with the authentic labels.
Our strategy is twofold: firstly, using CWCL to extract pertinent features to identify cleanly labeled samples, and secondly, progressively fine-tuning using these samples.
arXiv Detail & Related papers (2023-08-14T06:04:50Z) - NorMatch: Matching Normalizing Flows with Discriminative Classifiers for
Semi-Supervised Learning [8.749830466953584]
Semi-Supervised Learning (SSL) aims to learn a model using a tiny labeled set and massive amounts of unlabeled data.
In this work we introduce a new framework for SSL named NorMatch.
We demonstrate, through numerical and visual results, that NorMatch achieves state-of-the-art performance on several datasets.
arXiv Detail & Related papers (2022-11-17T15:39:18Z) - Leveraging Instance Features for Label Aggregation in Programmatic Weak
Supervision [75.1860418333995]
Programmatic Weak Supervision (PWS) has emerged as a widespread paradigm to synthesize training labels efficiently.
The core component of PWS is the label model, which infers true labels by aggregating the outputs of multiple noisy supervision sources as labeling functions.
Existing statistical label models typically rely only on the outputs of LF, ignoring the instance features when modeling the underlying generative process.
arXiv Detail & Related papers (2022-10-06T07:28:53Z) - Neighborhood Collective Estimation for Noisy Label Identification and
Correction [92.20697827784426]
Learning with noisy labels (LNL) aims at designing strategies to improve model performance and generalization by mitigating the effects of model overfitting to noisy labels.
Recent advances employ the predicted label distributions of individual samples to perform noise verification and noisy label correction, easily giving rise to confirmation bias.
We propose Neighborhood Collective Estimation, in which the predictive reliability of a candidate sample is re-estimated by contrasting it against its feature-space nearest neighbors.
arXiv Detail & Related papers (2022-08-05T14:47:22Z) - Gaussian Mixture Variational Autoencoder with Contrastive Learning for
Multi-Label Classification [27.043136219527767]
We propose a novel contrastive learning boosted multi-label prediction model.
By using contrastive learning in the supervised setting, we can exploit label information effectively.
We show that the learnt embeddings provide insights into the interpretation of label-label interactions.
arXiv Detail & Related papers (2021-12-02T04:23:34Z) - OpenMatch: Open-set Consistency Regularization for Semi-supervised
Learning with Outliers [71.08167292329028]
We propose a novel Open-set Semi-Supervised Learning (OSSL) approach called OpenMatch.
OpenMatch unifies FixMatch with novelty detection based on one-vs-all (OVA) classifiers.
It achieves state-of-the-art performance on three datasets, and even outperforms a fully supervised model in detecting outliers unseen in unlabeled data on CIFAR10.
arXiv Detail & Related papers (2021-05-28T23:57:15Z) - Training image classifiers using Semi-Weak Label Data [26.04162590798731]
In Multiple Instance learning (MIL), weak labels are provided at the bag level with only presence/absence information known.
This paper introduces a novel semi-weak label learning paradigm as a middle ground to mitigate the problem.
We propose a two-stage framework to address the problem of learning from semi-weak labels.
arXiv Detail & Related papers (2021-03-19T03:06:07Z) - GANs for learning from very high class conditional noisy labels [1.6516902135723865]
We use Generative Adversarial Networks (GANs) to design a class conditional label noise (CCN) robust scheme for binary classification.
It first generates a set of correctly labelled data points from noisy labelled data and 0.1% or 1% clean labels.
arXiv Detail & Related papers (2020-10-19T15:01:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.