Lower-bounded proper losses for weakly supervised classification

• URL: http://arxiv.org/abs/2103.02893v1
• Date: Thu, 4 Mar 2021 08:47:07 GMT
• Title: Lower-bounded proper losses for weakly supervised classification
• Authors: Shuhei M. Yoshida, Takashi Takenouchi, Masashi Sugiyama
• Abstract summary: We discuss the problem of weakly supervised learning of classification, in which instances are given weak labels. We derive a representation theorem for proper losses in supervised learning, which dualizes the Savage representation. We experimentally demonstrate the effectiveness of our proposed approach, as compared to improper or unbounded losses.
• Score: 73.974163801142
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper discusses the problem of weakly supervised learning of classification, in which instances are given weak labels that are produced by some label-corruption process. The goal is to derive conditions under which loss functions for weak-label learning are proper and lower-bounded -- two essential requirements for the losses used in class-probability estimation. To this end, we derive a representation theorem for proper losses in supervised learning, which dualizes the Savage representation. We use this theorem to characterize proper weak-label losses and find a condition for them to be lower-bounded. Based on these theoretical findings, we derive a novel regularization scheme called generalized logit squeezing, which makes any proper weak-label loss bounded from below, without losing properness. Furthermore, we experimentally demonstrate the effectiveness of our proposed approach, as compared to improper or unbounded losses. Those results highlight the importance of properness and lower-boundedness. The code is publicly available at https://github.com/yoshum/lower-bounded-proper-losses.

Related papers

• A fuzzy loss for ontology classification [0.0]
  We propose a fuzzy loss that combines label-based loss with terms penalising subsumption- or disjointness-violations. We show that the fuzzy loss is able to decrease the number of consistency violations by several orders of magnitude without decreasing the classification performance.
  arXiv  Detail & Related papers  (2024-05-03T13:20:37Z)
• Generating Unbiased Pseudo-labels via a Theoretically Guaranteed Chebyshev Constraint to Unify Semi-supervised Classification and Regression [57.17120203327993]
  threshold-to-pseudo label process (T2L) in classification uses confidence to determine the quality of label. In nature, regression also requires unbiased methods to generate high-quality labels. We propose a theoretically guaranteed constraint for generating unbiased labels based on Chebyshev's inequality.
  arXiv  Detail & Related papers  (2023-11-03T08:39:35Z)
• Partial-Label Regression [54.74984751371617]
  Partial-label learning is a weakly supervised learning setting that allows each training example to be annotated with a set of candidate labels. Previous studies on partial-label learning only focused on the classification setting where candidate labels are all discrete. In this paper, we provide the first attempt to investigate partial-label regression, where each training example is annotated with a set of real-valued candidate labels.
  arXiv  Detail & Related papers  (2023-06-15T09:02:24Z)
• Understanding Square Loss in Training Overparametrized Neural Network Classifiers [31.319145959402462]
  We contribute to the theoretical understanding of square loss in classification by systematically investigating how it performs for overparametrized neural networks. We consider two cases, according to whether classes are separable or not. In the general non-separable case, fast convergence rate is established for both misclassification rate and calibration error. The resulting margin is proven to be lower bounded away from zero, providing theoretical guarantees for robustness.
  arXiv  Detail & Related papers  (2021-12-07T12:12:30Z)
• Constrained Classification and Policy Learning [0.0]
  We study consistency of surrogate loss procedures under a constrained set of classifiers. We show that hinge losses are the only surrogate losses that preserve consistency in second-best scenarios.
  arXiv  Detail & Related papers  (2021-06-24T10:43:00Z)
• Leveraged Weighted Loss for Partial Label Learning [64.85763991485652]
  Partial label learning deals with data where each instance is assigned with a set of candidate labels, whereas only one of them is true. Despite many methodology studies on learning from partial labels, there still lacks theoretical understandings of their risk consistent properties. We propose a family of loss functions named textitd weighted (LW) loss, which for the first time introduces the leverage parameter $beta$ to consider the trade-off between losses on partial labels and non-partial ones.
  arXiv  Detail & Related papers  (2021-06-10T13:25:13Z)
• A Symmetric Loss Perspective of Reliable Machine Learning [87.68601212686086]
  We review how a symmetric loss can yield robust classification from corrupted labels in balanced error rate (BER) minimization. We demonstrate how the robust AUC method can benefit natural language processing in the problem where we want to learn only from relevant keywords.
  arXiv  Detail & Related papers  (2021-01-05T06:25:47Z)
• Calibrated Surrogate Losses for Adversarially Robust Classification [92.37268323142307]
  We show that no convex surrogate loss is respect with respect to adversarial 0-1 loss when restricted to linear models. We also show that if the underlying distribution satisfies the Massart's noise condition, convex losses can also be calibrated in the adversarial setting.
  arXiv  Detail & Related papers  (2020-05-28T02:40:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.