Combating Semantic Contamination in Learning with Label Noise

• URL: http://arxiv.org/abs/2412.11620v2
• Date: Tue, 17 Dec 2024 04:26:08 GMT
• Title: Combating Semantic Contamination in Learning with Label Noise
• Authors: Wenxiao Fan, Kan Li,
• Abstract summary: Noisy labels can negatively impact the performance of deep neural networks. One common solution is label refurbishment, which involves reconstructing noisy labels through predictions and distributions. These methods may introduce problematic semantic associations, a phenomenon that we identify as Semantic Contamination. We propose a novel method called Collaborative Cross Learning, which utilizes semi-supervised learning on refurbished labels to extract appropriate semantic associations from embeddings across views and models.
• Score: 4.1764890353795
• License:
• Abstract: Noisy labels can negatively impact the performance of deep neural networks. One common solution is label refurbishment, which involves reconstructing noisy labels through predictions and distributions. However, these methods may introduce problematic semantic associations, a phenomenon that we identify as Semantic Contamination. Through an analysis of Robust LR, a representative label refurbishment method, we found that utilizing the logits of views for refurbishment does not adequately balance the semantic information of individual classes. Conversely, using the logits of models fails to maintain consistent semantic relationships across models, which explains why label refurbishment methods frequently encounter issues related to Semantic Contamination. To address this issue, we propose a novel method called Collaborative Cross Learning, which utilizes semi-supervised learning on refurbished labels to extract appropriate semantic associations from embeddings across views and models. Experimental results show that our method outperforms existing approaches on both synthetic and real-world noisy datasets, effectively mitigating the impact of label noise and Semantic Contamination.

Related papers

• Compensation Learning in Semantic Segmentation [22.105356244579745]
  We propose Compensation Learning in Semanticscapes, a framework to identify and compensate ambiguities as well as label noise. We introduce a novel uncertainty branch for neural networks to induce the compensation bias only to relevant regions. Our method is employed into state-of-the-art segmentation frameworks and several experiments demonstrate that our proposed compensation learns inter-class relations.
  arXiv  Detail & Related papers  (2023-04-26T10:26:11Z)
• Category-Adaptive Label Discovery and Noise Rejection for Multi-label Image Recognition with Partial Positive Labels [78.88007892742438]
  Training multi-label models with partial positive labels (MLR-PPL) attracts increasing attention. Previous works regard unknown labels as negative and adopt traditional MLR algorithms. We propose to explore semantic correlation among different images to facilitate the MLR-PPL task.
  arXiv  Detail & Related papers  (2022-11-15T02:11:20Z)
• Learning Semantic Correspondence with Sparse Annotations [66.37298464505261]
  Finding dense semantic correspondence is a fundamental problem in computer vision. We propose a teacher-student learning paradigm for generating dense pseudo-labels. We also develop two novel strategies for denoising pseudo-labels.
  arXiv  Detail & Related papers  (2022-08-15T02:24:18Z)
• Transductive CLIP with Class-Conditional Contrastive Learning [68.51078382124331]
  We propose Transductive CLIP, a novel framework for learning a classification network with noisy labels from scratch. A class-conditional contrastive learning mechanism is proposed to mitigate the reliance on pseudo labels. ensemble labels is adopted as a pseudo label updating strategy to stabilize the training of deep neural networks with noisy labels.
  arXiv  Detail & Related papers  (2022-06-13T14:04:57Z)
• Part-based Pseudo Label Refinement for Unsupervised Person Re-identification [29.034390810078172]
  Unsupervised person re-identification (re-ID) aims at learning discriminative representations for person retrieval from unlabeled data. Recent techniques accomplish this task by using pseudo-labels, but these labels are inherently noisy and deteriorate the accuracy. We propose a novel Part-based Pseudo Label Refinement (PPLR) framework that reduces the label noise by employing the complementary relationship between global and part features.
  arXiv  Detail & Related papers  (2022-03-28T12:15:53Z)
• Resolving label uncertainty with implicit posterior models [71.62113762278963]
  We propose a method for jointly inferring labels across a collection of data samples. By implicitly assuming the existence of a generative model for which a differentiable predictor is the posterior, we derive a training objective that allows learning under weak beliefs.
  arXiv  Detail & Related papers  (2022-02-28T18:09:44Z)
• Label Noise in Adversarial Training: A Novel Perspective to Study Robust Overfitting [45.58217741522973]
  We show that label noise exists in adversarial training. Such label noise is due to the mismatch between the true label distribution of adversarial examples and the label inherited from clean examples. We propose a method to automatically calibrate the label to address the label noise and robust overfitting.
  arXiv  Detail & Related papers  (2021-10-07T01:15:06Z)
• Refining Pseudo Labels with Clustering Consensus over Generations for Unsupervised Object Re-identification [84.72303377833732]
  Unsupervised object re-identification targets at learning discriminative representations for object retrieval without any annotations. We propose to estimate pseudo label similarities between consecutive training generations with clustering consensus and refine pseudo labels with temporally propagated and ensembled pseudo labels. The proposed pseudo label refinery strategy is simple yet effective and can be seamlessly integrated into existing clustering-based unsupervised re-identification methods.
  arXiv  Detail & Related papers  (2021-06-11T02:42:42Z)
• Semi-supervised Relation Extraction via Incremental Meta Self-Training [56.633441255756075]
  Semi-Supervised Relation Extraction methods aim to leverage unlabeled data in addition to learning from limited samples. Existing self-training methods suffer from the gradual drift problem, where noisy pseudo labels on unlabeled data are incorporated during training. We propose a method called MetaSRE, where a Relation Label Generation Network generates quality assessment on pseudo labels by (meta) learning from the successful and failed attempts on Relation Classification Network as an additional meta-objective.
  arXiv  Detail & Related papers  (2020-10-06T03:54:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.