Robust Losses from Univariate Base Functions for Noisy-Label Learning

• URL: http://arxiv.org/abs/2607.16768v1
• Date: Sat, 18 Jul 2026 11:05:32 GMT
• Title: Robust Losses from Univariate Base Functions for Noisy-Label Learning
• Abstract summary: Learning with noisy labels is a fundamental problem in training reliable deep neural networks.<n>We propose a general framework that constructs robust multiclass losses from univariate base functions.<n>We show that the proposed losses achieve competitive or superior performance under various noise settings.
• Score: 18.268671025343696
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning with noisy labels is a fundamental problem in training reliable deep neural networks. Robust loss functions provide a direct and effective way to mitigate the adverse effects of label noise. However, most existing robust losses are designed directly at the level of the final multiclass objective, which makes it difficult to systematically characterize and extend their robustness properties. In this paper, we propose a general framework that constructs robust multiclass losses from univariate base functions. By defining mapping operators from base functions to multiclass losses, the robustness of the induced losses can be characterized through simple properties of the base functions. We develop two complementary construction schemes, Target Separation and Binary Reduction, corresponding to inter-class independent and inter-class dependent formulations, respectively. For both schemes, we analyze their symmetry and asymmetry properties and derive corresponding sufficient conditions, which provide theoretical criteria for noise-robust loss design. The proposed framework also provides a new route to constructing symmetric losses, serving as a complement to normalization-based symmetric loss designs. Extensive experiments on synthetic and real-world noisy-label benchmarks demonstrate that the proposed losses achieve competitive or superior performance under various noise settings.

Related papers

• UniRank: Unified Rank Allocation for Low-Rank LLM Compression [9.742379971053806]
  We formulate global low-rank allocation as a sorting-and-truncation pipeline.<n>We verify the strong correlation between high input-output cosine similarity and low effective rank.<n>We propose rank-preserving fine-tuning, which performs direct LoRA tuning on decomposed weights.
  arXiv  Detail & Related papers  (2026-06-20T03:10:33Z)
• Symmetrization of Loss Functions for Robust Training of Neural Networks in the Presence of Noisy Labels [48.06655334157014]
  We study a symmetrization method arising from the unique decomposition of any multi-class loss function into a symmetric component.<n>We show that this multi-class unhinged loss is the unique convex multi-class symmetric loss.<n>We then introduce SGCE and alpha-MAE, two loss functions that interpolate between the multi-class unhinged loss and the Mean Absolute Error.
  arXiv  Detail & Related papers  (2026-05-19T18:03:40Z)
• Importance-Guided Basis Selection for Low-Rank Decomposition of Large Language Models [9.690793619550654]
  We present Basis Selection with Importance (BSI), a principled low-rank compression framework that ranks and prunes bases.<n>We develop an efficient Hessian-diagonal estimator by adapting the Hutchinson randomized-probing method to loss curvature with symmetric parameter perturbations.<n>We provide a comprehensive theoretical analysis, including loss-increase bounds under basis pruning, explicit propagation of Hessian-diagonal estimation error into these bounds, variance characterization tied to the Hessian spectrum, high-probability sample-complexity guarantees for achieving a target estimation accuracy, and guidance on perturbation intensity.
  arXiv  Detail & Related papers  (2026-05-02T22:35:02Z)
• Variational Rectification Inference for Learning with Noisy Labels [74.85528327499662]
  We propose variational rectification inference (VRI) to formulate the adaptive rectification for loss functions.<n>VRI is constructed as a hierarchical Bayes by treating the rectifying vector as a latent variable.<n>By introducing the variational term in VRI, the conditional posterior is estimated accurately and avoids collapsing to a Dirac delta function.
  arXiv  Detail & Related papers  (2026-03-18T01:25:08Z)
• Robust low-rank estimation with multiple binary responses using pairwise AUC loss [0.0]
  Multiple binary responses arise in many modern data-analytic problems.<n>Low-rank models offer a natural way to encode latent dependence across tasks.<n>Existing methods for binary data are largely likelihood-based and focus on pointwise classification.
  arXiv  Detail & Related papers  (2026-01-13T15:00:10Z)
• The Multiclass Score-Oriented Loss (MultiSOL) on the Simplex [4.014524824655106]
  In supervised binary classification, score-oriented losses have been introduced with the aim of optimizing a chosen performance metric directly during the training phase.<n>In this paper, we use a recently introduced multidimensional threshold-based classification framework to extend such score-oriented losses to multiclass classification.<n>As also demonstrated by several classification experiments, this proposed family of losses is designed to preserve the main advantages observed in the binary setting.
  arXiv  Detail & Related papers  (2025-11-27T16:20:55Z)
• NDCG-Consistent Softmax Approximation with Accelerated Convergence [67.10365329542365]
  We propose novel loss formulations that align directly with ranking metrics.<n>We integrate the proposed RG losses with the highly efficient Alternating Least Squares (ALS) optimization method.<n> Empirical evaluations on real-world datasets demonstrate that our approach achieves comparable or superior ranking performance.
  arXiv  Detail & Related papers  (2025-06-11T06:59:17Z)
• Improve Noise Tolerance of Robust Loss via Noise-Awareness [60.34670515595074]
  We propose a meta-learning method which is capable of adaptively learning a hyper parameter prediction function, called Noise-Aware-Robust-Loss-Adjuster (NARL-Adjuster for brevity) Four SOTA robust loss functions are attempted to be integrated with our algorithm, and comprehensive experiments substantiate the general availability and effectiveness of the proposed method in both its noise tolerance and performance.
  arXiv  Detail & Related papers  (2023-01-18T04:54:58Z)
• Label Distributionally Robust Losses for Multi-class Classification: Consistency, Robustness and Adaptivity [55.29408396918968]
  We study a family of loss functions named label-distributionally robust (LDR) losses for multi-class classification. Our contributions include both consistency and robustness by establishing top-$k$ consistency of LDR losses for multi-class classification. We propose a new adaptive LDR loss that automatically adapts the individualized temperature parameter to the noise degree of class label of each instance.
  arXiv  Detail & Related papers  (2021-12-30T00:27:30Z)
• Asymmetric Loss Functions for Learning with Noisy Labels [82.50250230688388]
  We propose a new class of loss functions, namely textitasymmetric loss functions, which are robust to learning with noisy labels for various types of noise. Experimental results on benchmark datasets demonstrate that asymmetric loss functions can outperform state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-06T12:52:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.