Label Smoothing++: Enhanced Label Regularization for Training Neural Networks

• URL: http://arxiv.org/abs/2509.05307v1
• Date: Fri, 22 Aug 2025 23:11:43 GMT
• Title: Label Smoothing++: Enhanced Label Regularization for Training Neural Networks
• Authors: Sachin Chhabra, Hemanth Venkateswara, Baoxin Li,
• Abstract summary: Training neural networks with one-hot target labels often results in overconfidence and overfitting.<n>We propose a novel label regularization training strategy called Label Smoothing++.<n>Our approach uses a fixed label for the target class while enabling the network to learn the labels associated with non-target classes.
• Score: 8.189630642296416
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Training neural networks with one-hot target labels often results in overconfidence and overfitting. Label smoothing addresses this issue by perturbing the one-hot target labels by adding a uniform probability vector to create a regularized label. Although label smoothing improves the network's generalization ability, it assigns equal importance to all the non-target classes, which destroys the inter-class relationships. In this paper, we propose a novel label regularization training strategy called Label Smoothing++, which assigns non-zero probabilities to non-target classes and accounts for their inter-class relationships. Our approach uses a fixed label for the target class while enabling the network to learn the labels associated with non-target classes. Through extensive experiments on multiple datasets, we demonstrate how Label Smoothing++ mitigates overconfident predictions while promoting inter-class relationships and generalization capabilities.

Related papers

• Overcoming label shift with target-aware federated learning [10.355835466049092]
  Federated learning enables multiple actors to collaboratively train models without sharing private data.<n>A common reason is label shift -- that the label distributions differ between clients and the target domain.<n>We propose FedPALS, a principled and practical model aggregation scheme that adapts to label shifts to improve performance in the target domain.
  arXiv  Detail & Related papers  (2024-11-06T09:52:45Z)
• Federated Learning with Only Positive Labels by Exploring Label Correlations [78.59613150221597]
  Federated learning aims to collaboratively learn a model by using the data from multiple users under privacy constraints. In this paper, we study the multi-label classification problem under the federated learning setting. We propose a novel and generic method termed Federated Averaging by exploring Label Correlations (FedALC)
  arXiv  Detail & Related papers  (2024-04-24T02:22:50Z)
• Determined Multi-Label Learning via Similarity-Based Prompt [12.428779617221366]
  In multi-label classification, each training instance is associated with multiple class labels simultaneously. To alleviate this problem, a novel labeling setting termed textitDetermined Multi-Label Learning (DMLL) is proposed.
  arXiv  Detail & Related papers  (2024-03-25T07:08:01Z)
• Active Generalized Category Discovery [60.69060965936214]
  Generalized Category Discovery (GCD) endeavors to cluster unlabeled samples from both novel and old classes. We take the spirit of active learning and propose a new setting called Active Generalized Category Discovery (AGCD) Our method achieves state-of-the-art performance on both generic and fine-grained datasets.
  arXiv  Detail & Related papers  (2024-03-07T07:12:24Z)
• Imprecise Label Learning: A Unified Framework for Learning with Various Imprecise Label Configurations [91.67511167969934]
  imprecise label learning (ILL) is a framework for the unification of learning with various imprecise label configurations. We demonstrate that ILL can seamlessly adapt to partial label learning, semi-supervised learning, noisy label learning, and, more importantly, a mixture of these settings.
  arXiv  Detail & Related papers  (2023-05-22T04:50:28Z)
• Group is better than individual: Exploiting Label Topologies and Label Relations for Joint Multiple Intent Detection and Slot Filling [39.76268402567324]
  We construct a Heterogeneous Label Graph (HLG) containing two kinds of topologies. Label correlations are leveraged to enhance semantic-label interactions. We also propose the label-aware inter-dependent decoding mechanism to further exploit the label correlations for decoding.
  arXiv  Detail & Related papers  (2022-10-19T08:21:43Z)
• Transductive CLIP with Class-Conditional Contrastive Learning [68.51078382124331]
  We propose Transductive CLIP, a novel framework for learning a classification network with noisy labels from scratch. A class-conditional contrastive learning mechanism is proposed to mitigate the reliance on pseudo labels. ensemble labels is adopted as a pseudo label updating strategy to stabilize the training of deep neural networks with noisy labels.
  arXiv  Detail & Related papers  (2022-06-13T14:04:57Z)
• CLS: Cross Labeling Supervision for Semi-Supervised Learning [9.929229055862491]
  Cross Labeling Supervision ( CLS) is a framework that generalizes the typical pseudo-labeling process. CLS allows the creation of both pseudo and complementary labels to support both positive and negative learning.
  arXiv  Detail & Related papers  (2022-02-17T08:09:40Z)
• Instance-based Label Smoothing For Better Calibrated Classification Networks [3.388509725285237]
  Label smoothing is widely used in deep neural networks for multi-class classification. We take inspiration from both label smoothing and self-distillation. We propose two novel instance-based label smoothing approaches.
  arXiv  Detail & Related papers  (2021-10-11T15:33:23Z)
• A Study on the Autoregressive and non-Autoregressive Multi-label Learning [77.11075863067131]
  We propose a self-attention based variational encoder-model to extract the label-label and label-feature dependencies jointly. Our model can therefore be used to predict all labels in parallel while still including both label-label and label-feature dependencies.
  arXiv  Detail & Related papers  (2020-12-03T05:41:44Z)
• Learning Soft Labels via Meta Learning [3.4852307714135375]
  One-hot labels do not represent soft decision boundaries among concepts, and hence, models trained on them are prone to overfitting. We propose a framework, where we treat the labels as learnable parameters, and optimize them along with model parameters. We show that learned labels capture semantic relationship between classes, and thereby improve teacher models for the downstream task of distillation.
  arXiv  Detail & Related papers  (2020-09-20T18:42:13Z)
• Knowledge-Guided Multi-Label Few-Shot Learning for General Image Recognition [75.44233392355711]
  KGGR framework exploits prior knowledge of statistical label correlations with deep neural networks. It first builds a structured knowledge graph to correlate different labels based on statistical label co-occurrence. Then, it introduces the label semantics to guide learning semantic-specific features. It exploits a graph propagation network to explore graph node interactions.
  arXiv  Detail & Related papers  (2020-09-20T15:05:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.