Iterative Label Improvement: Robust Training by Confidence Based Filtering and Dataset Partitioning

• URL: http://arxiv.org/abs/2002.02705v3
• Date: Fri, 17 Jul 2020 10:13:54 GMT
• Title: Iterative Label Improvement: Robust Training by Confidence Based Filtering and Dataset Partitioning
• Authors: Christian Haase-Sch\"utz, Rainer Stal, Heinz Hertlein and Bernhard Sick
• Abstract summary: State-of-the-art, high capacity deep neural networks require large amounts of labelled training data. They are also highly susceptible to label errors in this data. We propose a novel meta training and labelling scheme that is able to use inexpensive unlabelled data.
• Score: 5.1293809610257775
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: State-of-the-art, high capacity deep neural networks not only require large amounts of labelled training data, they are also highly susceptible to label errors in this data, typically resulting in large efforts and costs and therefore limiting the applicability of deep learning. To alleviate this issue, we propose a novel meta training and labelling scheme that is able to use inexpensive unlabelled data by taking advantage of the generalization power of deep neural networks. We show experimentally that by solely relying on one network architecture and our proposed scheme of iterative training and prediction steps, both label quality and resulting model accuracy can be improved significantly. Our method achieves state-of-the-art results, while being architecture agnostic and therefore broadly applicable. Compared to other methods dealing with erroneous labels, our approach does neither require another network to be trained, nor does it necessarily need an additional, highly accurate reference label set. Instead of removing samples from a labelled set, our technique uses additional sensor data without the need for manual labelling. Furthermore, our approach can be used for semi-supervised learning.

Related papers

• Reducing Labeling Costs in Sentiment Analysis via Semi-Supervised Learning [0.0]
  This study explores label propagation in semi-supervised learning. We employ a transductive label propagation method based on the manifold assumption for text classification. By extending labels based on cosine proximity within a nearest neighbor graph from network embeddings, we combine unlabeled data into supervised learning.
  arXiv  Detail & Related papers  (2024-10-15T07:25:33Z)
• Boosting Semi-Supervised Learning by bridging high and low-confidence predictions [4.18804572788063]
  Pseudo-labeling is a crucial technique in semi-supervised learning (SSL) We propose a new method called ReFixMatch, which aims to utilize all of the unlabeled data during training.
  arXiv  Detail & Related papers  (2023-08-15T00:27:18Z)
• Weighted Distillation with Unlabeled Examples [15.825078347452024]
  Distillation with unlabeled examples is a popular and powerful method for training deep neural networks in settings where the amount of labeled data is limited. This paper proposes a principled approach for addressing this issue based on a ''debiasing'' reweighting of the student's loss function tailored to the distillation training paradigm.
  arXiv  Detail & Related papers  (2022-10-13T04:08:56Z)
• Semi-supervised Learning using Robust Loss [0.0]
  We suggest a semi-supervised training strategy for leveraging both manually labeled data and extra unlabeled data. In contrast to the existing approaches, we apply robust loss for the automated labeled data to compensate for the uneven data quality. We show that our proposed strategy improves the model performance by compensating for the uneven quality of labels in image classification.
  arXiv  Detail & Related papers  (2022-03-03T05:34:32Z)
• Debiased Pseudo Labeling in Self-Training [77.83549261035277]
  Deep neural networks achieve remarkable performances on a wide range of tasks with the aid of large-scale labeled datasets. To mitigate the requirement for labeled data, self-training is widely used in both academia and industry by pseudo labeling on readily-available unlabeled data. We propose Debiased, in which the generation and utilization of pseudo labels are decoupled by two independent heads.
  arXiv  Detail & Related papers  (2022-02-15T02:14:33Z)
• Towards Reducing Labeling Cost in Deep Object Detection [61.010693873330446]
  We propose a unified framework for active learning, that considers both the uncertainty and the robustness of the detector. Our method is able to pseudo-label the very confident predictions, suppressing a potential distribution drift.
  arXiv  Detail & Related papers  (2021-06-22T16:53:09Z)
• Self-Supervised Noisy Label Learning for Source-Free Unsupervised Domain Adaptation [87.60688582088194]
  We propose a novel Self-Supervised Noisy Label Learning method. Our method can easily achieve state-of-the-art results and surpass other methods by a very large margin.
  arXiv  Detail & Related papers  (2021-02-23T10:51:45Z)
• Mixed-Privacy Forgetting in Deep Networks [114.3840147070712]
  We show that the influence of a subset of the training samples can be removed from the weights of a network trained on large-scale image classification tasks. Inspired by real-world applications of forgetting techniques, we introduce a novel notion of forgetting in mixed-privacy setting. We show that our method allows forgetting without having to trade off the model accuracy.
  arXiv  Detail & Related papers  (2020-12-24T19:34:56Z)
• PseudoSeg: Designing Pseudo Labels for Semantic Segmentation [78.35515004654553]
  We present a re-design of pseudo-labeling to generate structured pseudo labels for training with unlabeled or weakly-labeled data. We demonstrate the effectiveness of the proposed pseudo-labeling strategy in both low-data and high-data regimes.
  arXiv  Detail & Related papers  (2020-10-19T17:59:30Z)
• COLAM: Co-Learning of Deep Neural Networks and Soft Labels via Alternating Minimization [60.07531696857743]
  Co-Learns DNNs and soft labels through Alternating Minimization of two objectives. We propose COLAM framework that Co-Learns DNNs and soft labels through Alternating Minimization of two objectives.
  arXiv  Detail & Related papers  (2020-04-26T17:50:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.