Rank-based Decomposable Losses in Machine Learning: A Survey

• URL: http://arxiv.org/abs/2207.08768v3
• Date: Fri, 14 Jul 2023 00:03:28 GMT
• Title: Rank-based Decomposable Losses in Machine Learning: A Survey
• Abstract summary: This survey provides a systematic and comprehensive review of rank-based decomposable losses in machine learning. We provide a new taxonomy of loss functions that follows the perspectives of aggregate loss and individual loss. We describe general formulas for rank-based aggregate losses and rank-based individual losses.
• Score: 47.547653738853406
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent works have revealed an essential paradigm in designing loss functions that differentiate individual losses vs. aggregate losses. The individual loss measures the quality of the model on a sample, while the aggregate loss combines individual losses/scores over each training sample. Both have a common procedure that aggregates a set of individual values to a single numerical value. The ranking order reflects the most fundamental relation among individual values in designing losses. In addition, decomposability, in which a loss can be decomposed into an ensemble of individual terms, becomes a significant property of organizing losses/scores. This survey provides a systematic and comprehensive review of rank-based decomposable losses in machine learning. Specifically, we provide a new taxonomy of loss functions that follows the perspectives of aggregate loss and individual loss. We identify the aggregator to form such losses, which are examples of set functions. We organize the rank-based decomposable losses into eight categories. Following these categories, we review the literature on rank-based aggregate losses and rank-based individual losses. We describe general formulas for these losses and connect them with existing research topics. We also suggest future research directions spanning unexplored, remaining, and emerging issues in rank-based decomposable losses.

Related papers

• Calibeating for general proper losses: A Bregman divergence approach [10.525701754548102]
  This work introduces a general framework for calibeating based on regret.<n>We consider a large family of proper losses that includes $$-Tsallis losses (for $in [1, 2]$) and Lipschitz losses.
  arXiv  Detail & Related papers  (2026-05-17T05:39:22Z)
• Improved Balanced Classification with Theoretically Grounded Loss Functions [41.69461814486466]
  Generalized Logit-Adjusted (GLA) loss functions and Generalized Class-Aware weighted (GCA) losses are studied.<n>We show that GLA losses are Bayes-consistent, but only $H$-consistent for complete (i.e., unbounded) hypothesis sets.<n>GCA losses are $H$-consistent for any hypothesis set that is bounded or complete, with $H$-consistency bounds that scale more favorably as $1/sqrtmathsf p_min$.
  arXiv  Detail & Related papers  (2025-12-30T02:34:02Z)
• Robust Loss Functions for Training Decision Trees with Noisy Labels [4.795403008763752]
  We consider training decision trees using noisily labeled data, focusing on loss functions that can lead to robust learning algorithms. First, we offer novel theoretical insights on the robustness of many existing loss functions in the context of decision tree learning. Second, we introduce a framework for constructing robust loss functions, called distribution losses.
  arXiv  Detail & Related papers  (2023-12-20T11:27:46Z)
• The Group Loss++: A deeper look into group loss for deep metric learning [65.19665861268574]
  Group Loss is a loss function based on a differentiable label-propagation method that enforces embedding similarity across all samples of a group. We show state-of-the-art results on clustering and image retrieval on four datasets, and present competitive results on two person re-identification datasets.
  arXiv  Detail & Related papers  (2022-04-04T14:09:58Z)
• Ensemble of Loss Functions to Improve Generalizability of Deep Metric Learning methods [0.609170287691728]
  We propose novel approaches to combine different losses built on top of a shared deep feature extractor. We evaluate our methods on some popular datasets from the machine vision domain in conventional Zero-Shot-Learning (ZSL) settings.
  arXiv  Detail & Related papers  (2021-07-02T15:19:46Z)
• Auto Seg-Loss: Searching Metric Surrogates for Semantic Segmentation [56.343646789922545]
  We propose to automate the design of metric-specific loss functions by searching differentiable surrogate losses for each metric. Experiments on PASCAL VOC and Cityscapes demonstrate that the searched surrogate losses outperform the manually designed loss functions consistently.
  arXiv  Detail & Related papers  (2020-10-15T17:59:08Z)
• Learning by Minimizing the Sum of Ranked Range [58.24935359348289]
  We introduce the sum of ranked range (SoRR) as a general approach to form learning objectives. A ranked range is a consecutive sequence of sorted values of a set of real numbers. We explore two applications in machine learning of the minimization of the SoRR framework, namely the AoRR aggregate loss for binary classification and the TKML individual loss for multi-label/multi-class classification.
  arXiv  Detail & Related papers  (2020-10-05T01:58:32Z)
• An Equivalence between Loss Functions and Non-Uniform Sampling in Experience Replay [72.23433407017558]
  We show that any loss function evaluated with non-uniformly sampled data can be transformed into another uniformly sampled loss function. Surprisingly, we find in some environments PER can be replaced entirely by this new loss function without impact to empirical performance.
  arXiv  Detail & Related papers  (2020-07-12T17:45:24Z)
• Rethinking preventing class-collapsing in metric learning with margin-based losses [81.22825616879936]
  Metric learning seeks embeddings where visually similar instances are close and dissimilar instances are apart. margin-based losses tend to project all samples of a class onto a single point in the embedding space. We propose a simple modification to the embedding losses such that each sample selects its nearest same-class counterpart in a batch.
  arXiv  Detail & Related papers  (2020-06-09T09:59:25Z)
• Segmentation Loss Odyssey [10.051935123214355]
  We present a systematic taxonomy to sort existing loss functions into four meaningful categories. We also explore the relationship between the traditional region-based and the more recent boundary-based loss functions.
  arXiv  Detail & Related papers  (2020-05-27T16:00:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.