Investigation of Training Label Error Impact on RNN-T

• URL: http://arxiv.org/abs/2112.00350v1
• Date: Wed, 1 Dec 2021 08:57:39 GMT
• Title: Investigation of Training Label Error Impact on RNN-T
• Authors: I-Fan Chen, Brian King, Jasha Droppo
• Abstract summary: We analyze impacts of different training label errors to RNN-T based ASR models. We suggest to design data pipelines for RNN-T with higher priority on reducing deletion label errors.
• Score: 8.470599402385302
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we propose an approach to quantitatively analyze impacts of different training label errors to RNN-T based ASR models. The result shows deletion errors are more harmful than substitution and insertion label errors in RNN-T training data. We also examined label error impact mitigation approaches on RNN-T and found that, though all the methods mitigate the label-error-caused degradation to some extent, they could not remove the performance gap between the models trained with and without the presence of label errors. Based on the analysis results, we suggest to design data pipelines for RNN-T with higher priority on reducing deletion label errors. We also find that ensuring high-quality training labels remains important, despite of the existence of the label error mitigation approaches.

Related papers

• DynClean: Training Dynamics-based Label Cleaning for Distantly-Supervised Named Entity Recognition [49.54155332262579]
  We propose a training dynamics-based label cleaning approach, which leverages the behavior of a model as training progresses. We also introduce an automatic threshold estimation strategy to locate the errors in distant labels. Our method outperforms numerous advanced DS-NER approaches across four datasets.
  arXiv  Detail & Related papers  (2025-04-06T20:54:42Z)
• Mitigating the Impact of Labeling Errors on Training via Rockafellian Relaxation [0.8741284539870512]
  We propose and study the implementation of Rockafellian Relaxation (RR) for neural network training. RR can enhance standard neural network methods to achieve robust performance across classification tasks. We find that RR can mitigate the effects of dataset corruption due to both (heavy) labeling error and/or adversarial perturbation.
  arXiv  Detail & Related papers  (2024-05-30T23:13:01Z)
• Parameter-tuning-free data entry error unlearning with adaptive selective synaptic dampening [51.34904967046097]
  We introduce an extension to the selective synaptic dampening unlearning method that removes the need for parameter tuning. We demonstrate the performance of this extension, adaptive selective synaptic dampening (ASSD) on various ResNet18 and Vision Transformer unlearning tasks. The application of this approach is particularly compelling in industrial settings, such as supply chain management.
  arXiv  Detail & Related papers  (2024-02-06T14:04:31Z)
• ERASE: Error-Resilient Representation Learning on Graphs for Label Noise Tolerance [53.73316938815873]
  We propose a method called ERASE (Error-Resilient representation learning on graphs for lAbel noiSe tolerancE) to learn representations with error tolerance. ERASE combines prototype pseudo-labels with propagated denoised labels and updates representations with error resilience. Our method can outperform multiple baselines with clear margins in broad noise levels and enjoy great scalability.
  arXiv  Detail & Related papers  (2023-12-13T17:59:07Z)
• Quantifying and mitigating the impact of label errors on model disparity metrics [14.225423850241675]
  We study the effect of label error on a model's disparity metrics. We find that group calibration and other metrics are sensitive to train-time and test-time label error. We present an approach to estimate the influence of a training input's label on a model's group disparity metric.
  arXiv  Detail & Related papers  (2023-10-04T02:18:45Z)
• Losses over Labels: Weakly Supervised Learning via Direct Loss Construction [71.11337906077483]
  Programmable weak supervision is a growing paradigm within machine learning. We propose Losses over Labels (LoL) as it creates losses directly from ofs without going through the intermediate step of a label. We show that LoL improves upon existing weak supervision methods on several benchmark text and image classification tasks.
  arXiv  Detail & Related papers  (2022-12-13T22:29:14Z)
• Pseudo-Label Noise Suppression Techniques for Semi-Supervised Semantic Segmentation [21.163070161951868]
  Semi-consuming learning (SSL) can reduce the need for large labelled datasets by incorporating unsupervised data into the training. Current SSL approaches use an initially supervised trained model to generate predictions for unlabelled images, called pseudo-labels. We use three mechanisms to control pseudo-label noise and errors.
  arXiv  Detail & Related papers  (2022-10-19T09:46:27Z)
• CTRL: Clustering Training Losses for Label Error Detection [4.49681473359251]
  In supervised machine learning, use of correct labels is extremely important to ensure high accuracy. We propose a novel framework, calledClustering TRaining Losses for label error detection. It detects label errors in two steps based on the observation that models learn clean and noisy labels in different ways.
  arXiv  Detail & Related papers  (2022-08-17T18:09:19Z)
• Automated Detection of Label Errors in Semantic Segmentation Datasets via Deep Learning and Uncertainty Quantification [5.279257531335345]
  We for the first time present a method for detecting label errors in semantic segmentation datasets with pixel-wise labels. Our approach is able to detect the vast majority of label errors while controlling the number of false label error detections.
  arXiv  Detail & Related papers  (2022-07-13T10:25:23Z)
• Boosting Semi-Supervised Face Recognition with Noise Robustness [54.342992887966616]
  This paper presents an effective solution to semi-supervised face recognition that is robust to the label noise aroused by the auto-labelling. We develop a semi-supervised face recognition solution, named Noise Robust Learning-Labelling (NRoLL), which is based on the robust training ability empowered by GN.
  arXiv  Detail & Related papers  (2021-05-10T14:43:11Z)
• Delving Deep into Label Smoothing [112.24527926373084]
  Label smoothing is an effective regularization tool for deep neural networks (DNNs) We present an Online Label Smoothing (OLS) strategy, which generates soft labels based on the statistics of the model prediction for the target category.
  arXiv  Detail & Related papers  (2020-11-25T08:03:11Z)
• Temporal Calibrated Regularization for Robust Noisy Label Learning [60.90967240168525]
  Deep neural networks (DNNs) exhibit great success on many tasks with the help of large-scale well annotated datasets. However, labeling large-scale data can be very costly and error-prone so that it is difficult to guarantee the annotation quality. We propose a Temporal Calibrated Regularization (TCR) in which we utilize the original labels and the predictions in the previous epoch together.
  arXiv  Detail & Related papers  (2020-07-01T04:48:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.