Label Denoising through Cross-Model Agreement

• URL: http://arxiv.org/abs/2308.13976v3
• Date: Tue, 19 Dec 2023 04:44:35 GMT
• Title: Label Denoising through Cross-Model Agreement
• Authors: Yu Wang, Xin Xin, Zaiqiao Meng, Joemon Jose, Fuli Feng
• Abstract summary: Memorizing noisy labels could affect the learning of the model, leading to sub-optimal performances. We propose a novel framework to learn robust machine-learning models from noisy labels.
• Score: 43.5145547124009
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learning from corrupted labels is very common in real-world machine-learning applications. Memorizing such noisy labels could affect the learning of the model, leading to sub-optimal performances. In this work, we propose a novel framework to learn robust machine-learning models from noisy labels. Through an empirical study, we find that different models make relatively similar predictions on clean examples, while the predictions on noisy examples vary much more across different models. Motivated by this observation, we propose \em denoising with cross-model agreement \em (DeCA) which aims to minimize the KL-divergence between the true label distributions parameterized by two machine learning models while maximizing the likelihood of data observation. We employ the proposed DeCA on both the binary label scenario and the multiple label scenario. For the binary label scenario, we select implicit feedback recommendation as the downstream task and conduct experiments with four state-of-the-art recommendation models on four datasets. For the multiple-label scenario, the downstream application is image classification on two benchmark datasets. Experimental results demonstrate that the proposed methods significantly improve the model performance compared with normal training and other denoising methods on both binary and multiple-label scenarios.

Related papers

• Dual-Decoupling Learning and Metric-Adaptive Thresholding for Semi-Supervised Multi-Label Learning [81.83013974171364]
  Semi-supervised multi-label learning (SSMLL) is a powerful framework for leveraging unlabeled data to reduce the expensive cost of collecting precise multi-label annotations. Unlike semi-supervised learning, one cannot select the most probable label as the pseudo-label in SSMLL due to multiple semantics contained in an instance. We propose a dual-perspective method to generate high-quality pseudo-labels.
  arXiv  Detail & Related papers  (2024-07-26T09:33:53Z)
• Pre-Trained Vision-Language Models as Partial Annotators [40.89255396643592]
  Pre-trained vision-language models learn massive data to model unified representations of images and natural languages. In this paper, we investigate a novel "pre-trained annotating - weakly-supervised learning" paradigm for pre-trained model application and experiment on image classification tasks.
  arXiv  Detail & Related papers  (2024-05-23T17:17:27Z)
• Pairwise Similarity Distribution Clustering for Noisy Label Learning [0.0]
  Noisy label learning aims to train deep neural networks using a large amount of samples with noisy labels. We propose a simple yet effective sample selection algorithm to divide the training samples into one clean set and another noisy set. Experimental results on various benchmark datasets, such as CIFAR-10, CIFAR-100 and Clothing1M, demonstrate significant improvements over state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-04-02T11:30:22Z)
• Virtual Category Learning: A Semi-Supervised Learning Method for Dense Prediction with Extremely Limited Labels [63.16824565919966]
  This paper proposes to use confusing samples proactively without label correction. A Virtual Category (VC) is assigned to each confusing sample in such a way that it can safely contribute to the model optimisation. Our intriguing findings highlight the usage of VC learning in dense vision tasks.
  arXiv  Detail & Related papers  (2023-12-02T16:23:52Z)
• Twin Contrastive Learning with Noisy Labels [45.31997043789471]
  We present TCL, a novel twin contrastive learning model to learn robust representations and handle noisy labels for classification. TCL achieves 7.5% improvements on CIFAR-10 with 90% noisy label -- an extremely noisy scenario.
  arXiv  Detail & Related papers  (2023-03-13T08:53:47Z)
• Asymmetric Co-teaching with Multi-view Consensus for Noisy Label Learning [15.690502285538411]
  We introduce our noisy-label learning approach, called Asymmetric Co-teaching (AsyCo) AsyCo produces more consistent divergent results of the co-teaching models. Experiments on synthetic and real-world noisy-label datasets show that AsyCo improves over current SOTA methods.
  arXiv  Detail & Related papers  (2023-01-01T04:10:03Z)
• Neighborhood Collective Estimation for Noisy Label Identification and Correction [92.20697827784426]
  Learning with noisy labels (LNL) aims at designing strategies to improve model performance and generalization by mitigating the effects of model overfitting to noisy labels. Recent advances employ the predicted label distributions of individual samples to perform noise verification and noisy label correction, easily giving rise to confirmation bias. We propose Neighborhood Collective Estimation, in which the predictive reliability of a candidate sample is re-estimated by contrasting it against its feature-space nearest neighbors.
  arXiv  Detail & Related papers  (2022-08-05T14:47:22Z)
• Gaussian Mixture Variational Autoencoder with Contrastive Learning for Multi-Label Classification [27.043136219527767]
  We propose a novel contrastive learning boosted multi-label prediction model. By using contrastive learning in the supervised setting, we can exploit label information effectively. We show that the learnt embeddings provide insights into the interpretation of label-label interactions.
  arXiv  Detail & Related papers  (2021-12-02T04:23:34Z)
• Learning Noise-Aware Encoder-Decoder from Noisy Labels by Alternating Back-Propagation for Saliency Detection [54.98042023365694]
  We propose a noise-aware encoder-decoder framework to disentangle a clean saliency predictor from noisy training examples. The proposed model consists of two sub-models parameterized by neural networks.
  arXiv  Detail & Related papers  (2020-07-23T18:47:36Z)
• Deep k-NN for Noisy Labels [55.97221021252733]
  We show that a simple $k$-nearest neighbor-based filtering approach on the logit layer of a preliminary model can remove mislabeled data and produce more accurate models than many recently proposed methods.
  arXiv  Detail & Related papers  (2020-04-26T05:15:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.