Leveraging Data Symmetries to Select an Optimal Subset of Training Data under Label Noise

• URL: http://arxiv.org/abs/2605.01874v1
• Date: Sun, 03 May 2026 13:37:17 GMT
• Title: Leveraging Data Symmetries to Select an Optimal Subset of Training Data under Label Noise
• Abstract summary: In noisy environments, there may exist a subset of the training data on which models can achieve performance comparable to training on a noise-free dataset.<n>We show that the performance of a classifier trained on a subset of a noisy dataset selected via cutstats is influenced by the accuracy of k-NN.
• Score: 8.979334145616019
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The performance of machine learning models often relies on large labeled datasets; however, data collected from diverse sources can contain label noise. Recent work has shown that, in noisy settings, there may exist a subset of the training data on which models can achieve performance comparable to training on a noise-free dataset. A widely used method for identifying such subsets is cutstats, which employs k-nearest neighbors (k-NN) to detect low-noise samples. However, its performance on high-dimensional data remains largely unexplored. In this work, we formally establish that the performance of a classifier trained on a subset of a noisy dataset selected via cutstats is influenced by the accuracy of k-NN. We further demonstrate that, in noisy environments, exploiting data invariance and knowledge of underlying symmetries can significantly enhance the performance of k-NN, bringing it closer to the Bayes optimal classifier even in high-dimensional regimes. Finally, we show that for real-world scenarios, where information about the underlying invariance is only partially known, learnt invariant representations can still facilitate the identification of near-optimal subsets.

Related papers

• Robust Classification under Noisy Labels: A Geometry-Aware Reliability Framework for Foundation Models [22.68107594048035]
  We present a two-stage framework to ensure robust classification in the presence of label noise without model retraining.<n>Recent work has shown that simple k-nearest neighbor approaches using an embedding derived from an FM can achieve good performance even in the presence of severe label noise.<n>In this paper, following a similar two-stage procedure, reliability estimation followed by reliability-weighted inference, we show that improved performance can be achieved by introducing geometry information.
  arXiv  Detail & Related papers  (2025-07-31T23:01:32Z)
• Meta-learning Representations for Learning from Multiple Annotators [40.886894995806955]
  We propose a meta-learning method for learning from multiple noisy annotators.<n>The proposed method embeds each example in tasks to a latent space by using a neural network.<n>We show the effectiveness of our method with real-world datasets with synthetic noise and real-world crowdsourcing datasets.
  arXiv  Detail & Related papers  (2025-06-12T00:58:37Z)
• Detect and Correct: A Selective Noise Correction Method for Learning with Noisy Labels [14.577138753507203]
  Falsely annotated samples, also known as noisy labels, can significantly harm the performance of deep learning models.<n>Two main approaches for learning with noisy labels are global noise estimation and data filtering.<n>Our method identifies potentially noisy samples based on their loss distribution.<n>We then apply a selection process to separate noisy and clean samples and learn a noise transition matrix to correct the loss for noisy samples while leaving the clean data unaffected.
  arXiv  Detail & Related papers  (2025-05-19T16:49:27Z)
• Enhancing Sample Utilization in Noise-Robust Deep Metric Learning With Subgroup-Based Positive-Pair Selection [84.78475642696137]
  The existence of noisy labels in real-world data negatively impacts the performance of deep learning models.<n>We propose a noise-robust DML framework with SubGroup-based Positive-pair Selection (SGPS)<n>SGPS constructs reliable positive pairs for noisy samples to enhance the sample utilization.
  arXiv  Detail & Related papers  (2025-01-19T14:41:55Z)
• Impact of Noisy Supervision in Foundation Model Learning [91.56591923244943]
  This paper is the first work to comprehensively understand and analyze the nature of noise in pre-training datasets.<n>We propose a tuning method (NMTune) to affine the feature space to mitigate the malignant effect of noise and improve generalization.
  arXiv  Detail & Related papers  (2024-03-11T16:22:41Z)
• Improving a Named Entity Recognizer Trained on Noisy Data with a Few Clean Instances [55.37242480995541]
  We propose to denoise noisy NER data with guidance from a small set of clean instances. Along with the main NER model we train a discriminator model and use its outputs to recalibrate the sample weights. Results on public crowdsourcing and distant supervision datasets show that the proposed method can consistently improve performance with a small guidance set.
  arXiv  Detail & Related papers  (2023-10-25T17:23:37Z)
• Learning Confident Classifiers in the Presence of Label Noise [5.551384206194696]
  This paper proposes a probabilistic model for noisy observations that allows us to build a confident classification and segmentation models.<n>Our experiments show that our algorithm outperforms state-of-the-art solutions for the considered classification and segmentation problems.
  arXiv  Detail & Related papers  (2023-01-02T04:27:25Z)
• Robust Meta-learning with Sampling Noise and Label Noise via Eigen-Reptile [78.1212767880785]
  meta-learner is prone to overfitting since there are only a few available samples. When handling the data with noisy labels, the meta-learner could be extremely sensitive to label noise. We present Eigen-Reptile (ER) that updates the meta- parameters with the main direction of historical task-specific parameters.
  arXiv  Detail & Related papers  (2022-06-04T08:48:02Z)
• The Optimal Noise in Noise-Contrastive Learning Is Not What You Think [80.07065346699005]
  We show that deviating from this assumption can actually lead to better statistical estimators. In particular, the optimal noise distribution is different from the data's and even from a different family.
  arXiv  Detail & Related papers  (2022-03-02T13:59:20Z)
• Generation and Analysis of Feature-Dependent Pseudo Noise for Training Deep Neural Networks [0.0]
  Training Deep neural networks (DNNs) on noisy labeled datasets is a challenging problem, because learning on mislabeled examples deteriorates the performance of the network. We propose an intuitive approach to creating feature-dependent noisy datasets by utilizing the training predictions of DNNs on clean datasets that also retain true label information. We conduct several experiments to establish that Pseudo noisy datasets resemble feature-dependent noisy datasets across different conditions.
  arXiv  Detail & Related papers  (2021-05-22T19:15:26Z)
• Tackling Instance-Dependent Label Noise via a Universal Probabilistic Model [80.91927573604438]
  This paper proposes a simple yet universal probabilistic model, which explicitly relates noisy labels to their instances. Experiments on datasets with both synthetic and real-world label noise verify that the proposed method yields significant improvements on robustness.
  arXiv  Detail & Related papers  (2021-01-14T05:43:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.