Revisiting Meta-Learning with Noisy Labels: Reweighting Dynamics and Theoretical Guarantees

• URL: http://arxiv.org/abs/2510.12209v1
• Date: Tue, 14 Oct 2025 07:00:12 GMT
• Title: Revisiting Meta-Learning with Noisy Labels: Reweighting Dynamics and Theoretical Guarantees
• Abstract summary: We propose a lightweight surrogate for meta-reweighting that integrates mean-centering, row shifting, and label-signed modulation.<n>Across synthetic and real noisy-label benchmarks, our method consistently outperforms strong reweighting/selection baselines.
• Score: 11.411266656183
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning with noisy labels remains challenging because over-parameterized networks memorize corrupted supervision. Meta-learning-based sample reweighting mitigates this by using a small clean subset to guide training, yet its behavior and training dynamics lack theoretical understanding. We provide a rigorous theoretical analysis of meta-reweighting under label noise and show that its training trajectory unfolds in three phases: (i) an alignment phase that amplifies examples consistent with a clean subset and suppresses conflicting ones; (ii) a filtering phase driving noisy example weights toward zero until the clean subset loss plateaus; and (iii) a post-filtering phase in which noise filtration becomes perturbation-sensitive. The mechanism is a similarity-weighted coupling between training and clean subset signals together with clean subset training loss contraction; in the post-filtering regime where the clean-subset loss is sufficiently small, the coupling term vanishes and meta-reweighting loses discriminatory power. Guided by this analysis, we propose a lightweight surrogate for meta-reweighting that integrates mean-centering, row shifting, and label-signed modulation, yielding more stable performance while avoiding expensive bi-level optimization. Across synthetic and real noisy-label benchmarks, our method consistently outperforms strong reweighting/selection baselines.

Related papers

• Robust Trajectory Distillation: Hybrid Reweighting Meets Teacher-Inspired Targets [51.490743278360064]
  We propose a trajectory-based DD framework that jointly suppresses noise and preserves transferable knowledge without relabeling or clean subsets.<n>It comprises two complementary components: Selective Guidance Reweighting (SGR) and Teacher-Inspired Auxiliary Targets (TIAT)
  arXiv  Detail & Related papers  (2026-06-29T06:24:53Z)
• Variational Rectification Inference for Learning with Noisy Labels [74.85528327499662]
  We propose variational rectification inference (VRI) to formulate the adaptive rectification for loss functions.<n>VRI is constructed as a hierarchical Bayes by treating the rectifying vector as a latent variable.<n>By introducing the variational term in VRI, the conditional posterior is estimated accurately and avoids collapsing to a Dirac delta function.
  arXiv  Detail & Related papers  (2026-03-18T01:25:08Z)
• Robust Self-Training with Closed-loop Label Correction for Learning from Noisy Labels [29.68163073890278]
  Training deep neural networks with noisy labels remains a significant challenge.<n>We propose a self-training label correction framework using decoupled bilevel optimization.<n>Our method employs noisy posterior simulation and intermediate features to transfer ground-truth knowledge.
  arXiv  Detail & Related papers  (2026-03-14T11:10:17Z)
• Deconstructing the Failure of Ideal Noise Correction: A Three-Pillar Diagnosis [82.06444754727156]
  We show that noise-correction methods still suffer from performance collapse during training.<n>This compellingly demonstrates that the failure is not fundamentally a $T$-estimation problem, but stems from a more deeply rooted flaw.<n>We provide a unified analysis that links three levels: macroscopic convergence states, microscopic optimisation dynamics, and information-theoretic limits on what can be learned from noisy labels.
  arXiv  Detail & Related papers  (2026-03-13T13:53:04Z)
• On the Learning Dynamics of Two-layer Linear Networks with Label Noise SGD [93.70725920710208]
  We study the learning dynamics of gradient descent with label noise.<n>We highlight the critical role of label noise in driving the transition from the lazy to the rich regime.<n>We extend these insights to Sharpness-Aware Minimization (SAM), showing that the principles governing label noise SGD also apply to broader optimization algorithms.
  arXiv  Detail & Related papers  (2026-03-11T04:26:54Z)
• Robust Unscented Kalman Filtering via Recurrent Meta-Adaptation of Sigma-Point Weights [0.0]
  This work introduces the Meta-Adaptive UKF (MA-UKF), a framework that reformulates sigma-point weight as a hyper parameter optimization problem.<n>Unlike standard adaptive filters that rely on instantaneous corrections, our approach employs a Recurrent Context to compress the history of measurement innovations into a compact latent embedding.<n> Numerical benchmarks on maneuvering targets demonstrate that the MA-UKF significantly outperforms standard baselines.
  arXiv  Detail & Related papers  (2026-03-04T18:27:59Z)
• The Malignant Tail: Spectral Segregation of Label Noise in Over-Parameterized Networks [0.0]
  We experimentally isolate the Malignant Tail, a failure mode where networks functionally segregate signal and noise.<n>We show that untrained networks actively segregate noise, allowing post-hoc Explicit Spectral Truncation to surgically prune the noise-dominated subspace.<n>Our findings suggest that under label noise, excess spectral capacity is not harmless redundancy but a latent structural liability.
  arXiv  Detail & Related papers  (2026-03-02T16:39:42Z)
• Combating Noisy Labels through Fostering Self- and Neighbor-Consistency [120.4394402099635]
  Label noise is pervasive in various real-world scenarios, posing challenges in supervised deep learning.<n>We propose a noise-robust method named Jo-SNC (textbfJoint sample selection and model regularization based on textbfSelf- and textbfNeighbor-textbfConsistency)<n>We design a self-adaptive, data-driven thresholding scheme to adjust per-class selection thresholds.
  arXiv  Detail & Related papers  (2026-01-19T07:55:29Z)
• Combating Noisy Labels via Dynamic Connection Masking [31.78040205653134]
  We propose a Dynamic Connection Masking (DCM) mechanism for both Multi-Layer Perceptron Networks (MLPs) and Kolmogorov-Arnold Networks (KANs)<n>Our approach can be seamlessly integrated into various noise-robust training methods to build more robust deep networks.
  arXiv  Detail & Related papers  (2025-08-13T10:51:46Z)
• Training More Robust Classification Model via Discriminative Loss and Gaussian Noise Injection [7.535952418691443]
  We introduce a loss function applied at the penultimate layer that explicitly enforces intra-class compactness.<n>We also propose a class-wise feature alignment mechanism that brings noisy data clusters closer to their clean counterparts.<n>Our approach significantly reinforces model robustness to various perturbations while maintaining high accuracy on clean data.
  arXiv  Detail & Related papers  (2024-05-28T18:10:45Z)
• Extracting Clean and Balanced Subset for Noisy Long-tailed Classification [66.47809135771698]
  We develop a novel pseudo labeling method using class prototypes from the perspective of distribution matching. By setting a manually-specific probability measure, we can reduce the side-effects of noisy and long-tailed data simultaneously. Our method can extract this class-balanced subset with clean labels, which brings effective performance gains for long-tailed classification with label noise.
  arXiv  Detail & Related papers  (2024-04-10T07:34:37Z)
• Rethinking Classifier Re-Training in Long-Tailed Recognition: A Simple Logits Retargeting Approach [102.0769560460338]
  We develop a simple logits approach (LORT) without the requirement of prior knowledge of the number of samples per class. Our method achieves state-of-the-art performance on various imbalanced datasets, including CIFAR100-LT, ImageNet-LT, and iNaturalist 2018.
  arXiv  Detail & Related papers  (2024-03-01T03:27:08Z)
• Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
  Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice. We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
  arXiv  Detail & Related papers  (2023-12-27T09:03:43Z)
• Latent Class-Conditional Noise Model [54.56899309997246]
  We introduce a Latent Class-Conditional Noise model (LCCN) to parameterize the noise transition under a Bayesian framework. We then deduce a dynamic label regression method for LCCN, whose Gibbs sampler allows us efficiently infer the latent true labels. Our approach safeguards the stable update of the noise transition, which avoids previous arbitrarily tuning from a mini-batch of samples.
  arXiv  Detail & Related papers  (2023-02-19T15:24:37Z)
• Robust Training under Label Noise by Over-parameterization [41.03008228953627]
  We propose a principled approach for robust training of over-parameterized deep networks in classification tasks where a proportion of training labels are corrupted. The main idea is yet very simple: label noise is sparse and incoherent with the network learned from clean data, so we model the noise and learn to separate it from the data. Remarkably, when trained using such a simple method in practice, we demonstrate state-of-the-art test accuracy against label noise on a variety of real datasets.
  arXiv  Detail & Related papers  (2022-02-28T18:50:10Z)
• S3: Supervised Self-supervised Learning under Label Noise [53.02249460567745]
  In this paper we address the problem of classification in the presence of label noise. In the heart of our method is a sample selection mechanism that relies on the consistency between the annotated label of a sample and the distribution of the labels in its neighborhood in the feature space. Our method significantly surpasses previous methods on both CIFARCIFAR100 with artificial noise and real-world noisy datasets such as WebVision and ANIMAL-10N.
  arXiv  Detail & Related papers  (2021-11-22T15:49:20Z)
• Consistency Regularization Can Improve Robustness to Label Noise [4.340338299803562]
  This paper empirically studies the relevance of consistency regularization for training-time robustness to noisy labels. We show that a simple loss function that encourages consistency improves the robustness of the models to label noise.
  arXiv  Detail & Related papers  (2021-10-04T08:15:08Z)
• Training Classifiers that are Universally Robust to All Label Noise Levels [91.13870793906968]
  Deep neural networks are prone to overfitting in the presence of label noise. We propose a distillation-based framework that incorporates a new subcategory of Positive-Unlabeled learning. Our framework generally outperforms at medium to high noise levels.
  arXiv  Detail & Related papers  (2021-05-27T13:49:31Z)
• Meta Transition Adaptation for Robust Deep Learning with Noisy Labels [61.8970957519509]
  This study proposes a new meta-transition-learning strategy for the task. Specifically, through the sound guidance of a small set of meta data with clean labels, the noise transition matrix and the classifier parameters can be mutually ameliorated. Our method can more accurately extract the transition matrix, naturally following its more robust performance than prior arts.
  arXiv  Detail & Related papers  (2020-06-10T07:27:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.