PLReMix: Combating Noisy Labels with Pseudo-Label Relaxed Contrastive
Representation Learning
- URL: http://arxiv.org/abs/2402.17589v1
- Date: Tue, 27 Feb 2024 15:22:20 GMT
- Title: PLReMix: Combating Noisy Labels with Pseudo-Label Relaxed Contrastive
Representation Learning
- Authors: Xiaoyu Liu, Beitong Zhou, Cheng Cheng
- Abstract summary: We propose an end-to-end PLReMix framework that avoids the complicated pipeline by introducing a Pseudo-Label Relaxed (PLR) contrastive loss.
PLR loss constructs a reliable negative set of each sample by filtering out its inappropriate negative pairs that overlap at the top k indices of prediction probabilities.
Our proposed PLR loss is scalable, which can be easily integrated into other LNL methods and boost their performance.
- Score: 5.962428976778709
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recently, the application of Contrastive Representation Learning (CRL) in
learning with noisy labels (LNL) has shown promising advancements due to its
remarkable ability to learn well-distributed representations for better
distinguishing noisy labels. However, CRL is mainly used as a pre-training
technique, leading to a complicated multi-stage training pipeline. We also
observed that trivially combining CRL with supervised LNL methods decreases
performance. Using different images from the same class as negative pairs in
CRL creates optimization conflicts between CRL and the supervised loss. To
address these two issues, we propose an end-to-end PLReMix framework that
avoids the complicated pipeline by introducing a Pseudo-Label Relaxed (PLR)
contrastive loss to alleviate the conflicts between losses. This PLR loss
constructs a reliable negative set of each sample by filtering out its
inappropriate negative pairs that overlap at the top k indices of prediction
probabilities, leading to more compact semantic clusters than vanilla CRL.
Furthermore, a two-dimensional Gaussian Mixture Model (GMM) is adopted to
distinguish clean and noisy samples by leveraging semantic information and
model outputs simultaneously, which is expanded on the previously widely used
one-dimensional form. The PLR loss and a semi-supervised loss are
simultaneously applied to train on the GMM divided clean and noisy samples.
Experiments on multiple benchmark datasets demonstrate the effectiveness of the
proposed method. Our proposed PLR loss is scalable, which can be easily
integrated into other LNL methods and boost their performance. Codes will be
available.
Related papers
- Decoupled Prototype Learning for Reliable Test-Time Adaptation [50.779896759106784]
Test-time adaptation (TTA) is a task that continually adapts a pre-trained source model to the target domain during inference.
One popular approach involves fine-tuning model with cross-entropy loss according to estimated pseudo-labels.
This study reveals that minimizing the classification error of each sample causes the cross-entropy loss's vulnerability to label noise.
We propose a novel Decoupled Prototype Learning (DPL) method that features prototype-centric loss computation.
arXiv Detail & Related papers (2024-01-15T03:33:39Z) - Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice.
We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
arXiv Detail & Related papers (2023-12-27T09:03:43Z) - Learning with Noisy Labels Using Collaborative Sample Selection and
Contrastive Semi-Supervised Learning [76.00798972439004]
Collaborative Sample Selection (CSS) removes noisy samples from identified clean set.
We introduce a co-training mechanism with a contrastive loss in semi-supervised learning.
arXiv Detail & Related papers (2023-10-24T05:37:20Z) - Learning in Imperfect Environment: Multi-Label Classification with
Long-Tailed Distribution and Partial Labels [53.68653940062605]
We introduce a novel task, Partial labeling and Long-Tailed Multi-Label Classification (PLT-MLC)
We find that most LT-MLC and PL-MLC approaches fail to solve the degradation-MLC.
We propose an end-to-end learning framework: textbfCOrrection $rightarrow$ textbfModificattextbfIon $rightarrow$ balantextbfCe.
arXiv Detail & Related papers (2023-04-20T20:05:08Z) - Learning Compact Features via In-Training Representation Alignment [19.273120635948363]
In each epoch, the true gradient of the loss function is estimated using a mini-batch sampled from the training set.
We propose In-Training Representation Alignment (ITRA) that explicitly aligns feature distributions of two different mini-batches with a matching loss.
We also provide a rigorous analysis of the desirable effects of the matching loss on feature representation learning.
arXiv Detail & Related papers (2022-11-23T22:23:22Z) - Positive-Negative Equal Contrastive Loss for Semantic Segmentation [8.664491798389662]
Previous works commonly design plug-and-play modules and structural losses to effectively extract and aggregate the global context.
We propose Positive-Negative Equal contrastive loss (PNE loss), which increases the latent impact of positive embedding on the anchor and treats the positive as well as negative sample pairs equally.
We conduct comprehensive experiments and achieve state-of-the-art performance on two benchmark datasets.
arXiv Detail & Related papers (2022-07-04T13:51:29Z) - Two-stage Training for Learning from Label Proportions [18.78148397471913]
Learning from label proportions (LLP) aims at learning an instance-level classifier with label proportions in grouped training data.
We introduce the mixup strategy and symmetric crossentropy to further reduce the label noise.
Our framework is model-agnostic, and demonstrates compelling performance improvement in extensive experiments.
arXiv Detail & Related papers (2021-05-22T03:55:35Z) - Contrastive Attraction and Contrastive Repulsion for Representation
Learning [131.72147978462348]
Contrastive learning (CL) methods learn data representations in a self-supervision manner, where the encoder contrasts each positive sample over multiple negative samples.
Recent CL methods have achieved promising results when pretrained on large-scale datasets, such as ImageNet.
We propose a doubly CL strategy that separately compares positive and negative samples within their own groups, and then proceeds with a contrast between positive and negative groups.
arXiv Detail & Related papers (2021-05-08T17:25:08Z) - Modal Regression based Structured Low-rank Matrix Recovery for
Multi-view Learning [70.57193072829288]
Low-rank Multi-view Subspace Learning has shown great potential in cross-view classification in recent years.
Existing LMvSL based methods are incapable of well handling view discrepancy and discriminancy simultaneously.
We propose Structured Low-rank Matrix Recovery (SLMR), a unique method of effectively removing view discrepancy and improving discriminancy.
arXiv Detail & Related papers (2020-03-22T03:57:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.