Symmetrization of Loss Functions for Robust Training of Neural Networks in the Presence of Noisy Labels

• URL: http://arxiv.org/abs/2605.20347v1
• Date: Tue, 19 May 2026 18:03:40 GMT
• Title: Symmetrization of Loss Functions for Robust Training of Neural Networks in the Presence of Noisy Labels
• Abstract summary: We study a symmetrization method arising from the unique decomposition of any multi-class loss function into a symmetric component.<n>We show that this multi-class unhinged loss is the unique convex multi-class symmetric loss.<n>We then introduce SGCE and alpha-MAE, two loss functions that interpolate between the multi-class unhinged loss and the Mean Absolute Error.
• Score: 48.06655334157014
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Labeling a training set is often expensive and susceptible to errors, making the design of robust loss functions for label noise an important problem. The symmetry condition provides theoretical guarantees for robustness to such noise. In this work, we study a symmetrization method arising from the unique decomposition of any multi-class loss function into a symmetric component and a class-insensitive term. In particular, symmetrizing the cross-entropy loss leads to a linear multi-class extension of the unhinged loss. Unlike in the binary case, the multi-class version must have specific coefficients in order to satisfy the symmetry condition. Under suitable assumptions, we show that this multi-class unhinged loss is the unique convex multi-class symmetric loss. We also show that it has a fundamental local role: the linear approximation of any symmetric loss around score vectors with equal components is equivalent to the multi-class unhinged loss. We then introduce SGCE and alpha-MAE, two loss functions that interpolate between the multi-class unhinged loss and the Mean Absolute Error while allowing control of the beta-smoothness of the loss. Experiments on standard noisy-label benchmarks show competitive performance compared with existing robust loss functions.

Related papers

• Robust Losses from Univariate Base Functions for Noisy-Label Learning [18.268671025343696]
  Learning with noisy labels is a fundamental problem in training reliable deep neural networks.<n>We propose a general framework that constructs robust multiclass losses from univariate base functions.<n>We show that the proposed losses achieve competitive or superior performance under various noise settings.
  arXiv  Detail & Related papers  (2026-07-18T11:05:32Z)
• Variation-Bounded Loss for Noise-Tolerant Learning [105.20373602308284]
  We introduce the Variation Ratio as a novel property related to the robustness of loss functions.<n>We propose a new family of robust loss functions, termed Variation-Bounded Loss (VBL), which is characterized by a bounded variation ratio.
  arXiv  Detail & Related papers  (2025-11-15T10:15:29Z)
• Lower Bounds on Adversarial Robustness for Multiclass Classification with General Loss Functions [4.562056072136493]
  We consider adversarially robust classification in a multiclass setting under arbitrary loss functions.<n>We derive dual and barycentric reformulations of the corresponding learner-agnostic robust risk problem.
  arXiv  Detail & Related papers  (2025-10-02T12:42:36Z)
• Joint Asymmetric Loss for Learning with Noisy Labels [95.14298444251044]
  symmetric losses usually suffer from the underfitting issue due to the overly strict constraint.<n>Within APL, symmetric losses have been successfully extended, yielding advanced robust loss functions.<n>We introduce a novel robust loss framework termed Joint Asymmetric Loss (JAL)
  arXiv  Detail & Related papers  (2025-07-23T16:57:43Z)
• Label Distributionally Robust Losses for Multi-class Classification: Consistency, Robustness and Adaptivity [55.29408396918968]
  We study a family of loss functions named label-distributionally robust (LDR) losses for multi-class classification. Our contributions include both consistency and robustness by establishing top-$k$ consistency of LDR losses for multi-class classification. We propose a new adaptive LDR loss that automatically adapts the individualized temperature parameter to the noise degree of class label of each instance.
  arXiv  Detail & Related papers  (2021-12-30T00:27:30Z)
• Asymmetric Loss Functions for Learning with Noisy Labels [82.50250230688388]
  We propose a new class of loss functions, namely textitasymmetric loss functions, which are robust to learning with noisy labels for various types of noise. Experimental results on benchmark datasets demonstrate that asymmetric loss functions can outperform state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-06T12:52:48Z)
• A Symmetric Loss Perspective of Reliable Machine Learning [87.68601212686086]
  We review how a symmetric loss can yield robust classification from corrupted labels in balanced error rate (BER) minimization. We demonstrate how the robust AUC method can benefit natural language processing in the problem where we want to learn only from relevant keywords.
  arXiv  Detail & Related papers  (2021-01-05T06:25:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.