Calibeating for general proper losses: A Bregman divergence approach

• URL: http://arxiv.org/abs/2605.17269v1
• Date: Sun, 17 May 2026 05:39:22 GMT
• Title: Calibeating for general proper losses: A Bregman divergence approach
• Abstract summary: This work introduces a general framework for calibeating based on regret.<n>We consider a large family of proper losses that includes $$-Tsallis losses (for $in [1, 2]$) and Lipschitz losses.
• Score: 10.525701754548102
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work introduces a general framework for calibeating based on regret minimization. As compared to Foster and Hart's seminal calibeating work which had specialized treatments of Brier score (squared loss) and log loss, we consider a large family of proper losses that includes $α$-Tsallis losses (for $α\in [1, 2]$) and Lipschitz losses. Our results for Tsallis losses also hold for an unscaled version of Tsallis loss that recovers log loss. Our analysis is oriented around the Bregman divergence view of a proper loss. Technically, our results for the family of Tsallis losses that we consider are U-calibration results, simultaneously obtaining logarithmic regret for all losses in this family while having a weaker dependence on the dimension compared to previous results. Of potential independent interest, we also show a new regret equality for the regret of Be The Regularized Leader. This regret equality holds for general proper losses and itself is based on two results related to online updating formulas for the generalized variance, the latter being a previously introduced generalization of variance based on Bregman divergences.

Related papers

• Calibeating Made Simple [75.91049650502048]
  We show that calibeating is minimax-equivalent to regret.<n>We also prove that multi-calibeating is minimax-equivalent to the combination of calibeating and the classical expert problem.<n>For binary predictions, our result gives the first calibrated algorithm that at the same time also achieves the optimal calibeating rate.
  arXiv  Detail & Related papers  (2026-03-23T16:28:07Z)
• Improved Balanced Classification with Theoretically Grounded Loss Functions [41.69461814486466]
  Generalized Logit-Adjusted (GLA) loss functions and Generalized Class-Aware weighted (GCA) losses are studied.<n>We show that GLA losses are Bayes-consistent, but only $H$-consistent for complete (i.e., unbounded) hypothesis sets.<n>GCA losses are $H$-consistent for any hypothesis set that is bounded or complete, with $H$-consistency bounds that scale more favorably as $1/sqrtmathsf p_min$.
  arXiv  Detail & Related papers  (2025-12-30T02:34:02Z)
• Rectifying Regression in Reinforcement Learning [51.28909745713678]
  We show that mean absolute error is a better prediction objective than the traditional mean squared error for controlling the learned policy's suboptimality gap.<n>We present results that different loss functions are better aligned with these different regression objectives.
  arXiv  Detail & Related papers  (2025-10-01T13:32:07Z)
• Bias-variance decompositions: the exclusive privilege of Bregman divergences [1.4383187348276911]
  We study continuous, nonnegative loss functions that satisfy the identity of indiscernibles.<n>A $g$-Bregman divergence can be transformed into a standard Bregman divergence through an invertible change of variables.
  arXiv  Detail & Related papers  (2025-01-30T18:52:44Z)
• LEARN: An Invex Loss for Outlier Oblivious Robust Online Optimization [56.67706781191521]
  An adversary can introduce outliers by corrupting loss functions in an arbitrary number of k, unknown to the learner. We present a robust online rounds optimization framework, where an adversary can introduce outliers by corrupting loss functions in an arbitrary number of k, unknown.
  arXiv  Detail & Related papers  (2024-08-12T17:08:31Z)
• Cross-Entropy Loss Functions: Theoretical Analysis and Applications [27.3569897539488]
  We present a theoretical analysis of a broad family of loss functions, that includes cross-entropy (or logistic loss), generalized cross-entropy, the mean absolute error and other cross-entropy-like loss functions. We show that these loss functions are beneficial in the adversarial setting by proving that they admit $H$-consistency bounds. This leads to new adversarial robustness algorithms that consist of minimizing a regularized smooth adversarial comp-sum loss.
  arXiv  Detail & Related papers  (2023-04-14T17:58:23Z)
• Rank-based Decomposable Losses in Machine Learning: A Survey [47.547653738853406]
  This survey provides a systematic and comprehensive review of rank-based decomposable losses in machine learning. We provide a new taxonomy of loss functions that follows the perspectives of aggregate loss and individual loss. We describe general formulas for rank-based aggregate losses and rank-based individual losses.
  arXiv  Detail & Related papers  (2022-07-18T17:21:01Z)
• Focal and Efficient IOU Loss for Accurate Bounding Box Regression [63.14659624634066]
  In object detection, bounding box regression (BBR) is a crucial step that determines the object localization performance. Most previous loss functions for BBR have two main drawbacks: (i) Both $ell_n$-norm and IOU-based loss functions are inefficient to depict the objective of BBR, which leads to slow convergence and inaccurate regression results.
  arXiv  Detail & Related papers  (2021-01-20T14:33:58Z)
• Non-exponentially weighted aggregation: regret bounds for unbounded loss functions [8.22379888383833]
  We tackle the problem of online optimization with a general, possibly unbounded, loss function. It is well known that when the loss is bounded, the exponentially weighted aggregation strategy (EWA) leads to a regret in $sqrtT$ after $T$ steps.
  arXiv  Detail & Related papers  (2020-09-07T11:09:08Z)
• Supervised Learning: No Loss No Cry [51.07683542418145]
  Supervised learning requires the specification of a loss function to minimise. This paper revisits the sc SLIsotron algorithm of Kakade et al. (2011) through a novel lens. We show how it provides a principled procedure for learning the loss.
  arXiv  Detail & Related papers  (2020-02-10T05:30:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.