Unified Binary and Multiclass Margin-Based Classification

• URL: http://arxiv.org/abs/2311.17778v2
• Date: Fri, 17 May 2024 08:46:15 GMT
• Title: Unified Binary and Multiclass Margin-Based Classification
• Authors: Yutong Wang, Clayton Scott,
• Abstract summary: We show that a broad range of multiclass loss functions, including many popular ones, can be expressed in the relative margin form. We then analyze the class of Fenchel-Young losses, and expand the set of these losses that are known to be classification-calibrated.
• Score: 27.28814893730265
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The notion of margin loss has been central to the development and analysis of algorithms for binary classification. To date, however, there remains no consensus as to the analogue of the margin loss for multiclass classification. In this work, we show that a broad range of multiclass loss functions, including many popular ones, can be expressed in the relative margin form, a generalization of the margin form of binary losses. The relative margin form is broadly useful for understanding and analyzing multiclass losses as shown by our prior work (Wang and Scott, 2020, 2021). To further demonstrate the utility of this way of expressing multiclass losses, we use it to extend the seminal result of Bartlett et al. (2006) on classification-calibration of binary margin losses to multiclass. We then analyze the class of Fenchel-Young losses, and expand the set of these losses that are known to be classification-calibrated.

Related papers

• The Implicit Bias of Gradient Descent on Separable Multiclass Data [38.05903703331163]
  We employ the framework of Permutation Equivariant and Relative Margin-based (PERM) losses to introduce a multiclass extension of the exponential tail property. Our proof techniques closely mirror those of the binary case, thus illustrating the power of the PERM framework for bridging the binary-multiclass gap.
  arXiv  Detail & Related papers  (2024-11-02T19:39:21Z)
• A Universal Growth Rate for Learning with Smooth Surrogate Losses [30.389055604165222]
  We prove a square-root growth rate near zero for smooth margin-based surrogate losses in binary classification. We extend this analysis to multi-class classification with a series of novel results.
  arXiv  Detail & Related papers  (2024-05-09T17:59:55Z)
• Learning Towards the Largest Margins [83.7763875464011]
  Loss function should promote the largest possible margins for both classes and samples. Not only does this principled framework offer new perspectives to understand and interpret existing margin-based losses, but it can guide the design of new tools.
  arXiv  Detail & Related papers  (2022-06-23T10:03:03Z)
• Multiclass learning with margin: exponential rates with no bias-variance trade-off [16.438523317718694]
  We study the behavior of error bounds for multiclass classification under suitable margin conditions. Different convergence rates can be obtained in correspondence of different margin assumptions.
  arXiv  Detail & Related papers  (2022-02-03T18:57:27Z)
• DropLoss for Long-Tail Instance Segmentation [56.162929199998075]
  We develop DropLoss, a novel adaptive loss to compensate for the imbalance between rare and frequent categories. We show state-of-the-art mAP across rare, common, and frequent categories on the LVIS dataset.
  arXiv  Detail & Related papers  (2021-04-13T17:59:22Z)
• Margin-Based Transfer Bounds for Meta Learning with Deep Feature Embedding [67.09827634481712]
  We leverage margin theory and statistical learning theory to establish three margin-based transfer bounds for meta-learning based multiclass classification (MLMC) These bounds reveal that the expected error of a given classification algorithm for a future task can be estimated with the average empirical error on a finite number of previous tasks. Experiments on three benchmarks show that these margin-based models still achieve competitive performance.
  arXiv  Detail & Related papers  (2020-12-02T23:50:51Z)
• Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets [98.74153364118898]
  We present a new loss function called Distribution-Balanced Loss for the multi-label recognition problems that exhibit long-tailed class distributions. The Distribution-Balanced Loss tackles these issues through two key modifications to the standard binary cross-entropy loss. Experiments on both Pascal VOC and COCO show that the models trained with this new loss function achieve significant performance gains.
  arXiv  Detail & Related papers  (2020-07-19T11:50:10Z)
• Rethinking preventing class-collapsing in metric learning with margin-based losses [81.22825616879936]
  Metric learning seeks embeddings where visually similar instances are close and dissimilar instances are apart. margin-based losses tend to project all samples of a class onto a single point in the embedding space. We propose a simple modification to the embedding losses such that each sample selects its nearest same-class counterpart in a batch.
  arXiv  Detail & Related papers  (2020-06-09T09:59:25Z)
• Negative Margin Matters: Understanding Margin in Few-shot Classification [72.85978953262004]
  This paper introduces a negative margin loss to metric learning based few-shot learning methods. The negative margin loss significantly outperforms regular softmax loss, and state-of-the-art accuracy on three standard few-shot classification benchmarks.
  arXiv  Detail & Related papers  (2020-03-26T17:59:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.