Loss-Shift Transfer via Bayes Quotients

• URL: http://arxiv.org/abs/2606.13178v1
• Date: Thu, 11 Jun 2026 10:46:19 GMT
• Title: Loss-Shift Transfer via Bayes Quotients
• Abstract summary: A loss determines which information in (X) is Bayes-relevant.<n>Two losses may therefore require different representations even under the same joint law (P(X,Y))<n>Experiments in controlled, learned, synthetic-image, and real-image settings show the predicted effect.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Transfer learning is usually studied as a consequence of distribution shift. This paper identifies an orthogonal failure mode in which the data distribution is fixed and the loss changes. This setting is called \emph{loss shift}. A loss determines which information in \(X\) is Bayes-relevant, and two losses may therefore require different representations even under the same joint law \(P(X,Y)\). The idea is formalized using Bayes quotients, which allow losses to be ordered by refinement. In the Bayes-quotient formulation, strict refinement gives an immediate qualitative obstruction. A source-minimal representation for a coarser loss is insufficient for a strictly finer target loss. For finite-output log loss, this obstruction becomes an exact quantitative identity. The excess risk is the conditional information about \(Y\) discarded by the representation. Experiments in controlled, learned, synthetic-image, and real-image settings show the predicted effect, i.e., classification-equivalent representations can have different optimal log-loss performance under a fixed data distribution.

Related papers

• Bayes-Sufficient Representations in Supervised Learning [0.0]
  This work asks what relevance means for a fixed supervised decision problem.<n>A representation is defined to be Bayes-sufficient for a joint distribution and loss if some prediction head can use it to implement a Bayes-optimal action rule.<n>For a fixed supervised problem, the distribution and the loss determine the Bayes action, the Bayes action determines the quotient, and the quotient determines the minimal information required for Bayes-optimal prediction.
  arXiv  Detail & Related papers  (2026-06-02T07:13:31Z)
• Bias-variance decompositions: the exclusive privilege of Bregman divergences [1.4383187348276911]
  We study continuous, nonnegative loss functions that satisfy the identity of indiscernibles.<n>A $g$-Bregman divergence can be transformed into a standard Bregman divergence through an invertible change of variables.
  arXiv  Detail & Related papers  (2025-01-30T18:52:44Z)
• EnsLoss: Stochastic Calibrated Loss Ensembles for Preventing Overfitting in Classification [1.3778851745408134]
  We propose a novel ensemble method, namely EnsLoss, to combine loss functions within the Empirical risk minimization framework. We first transform the CC conditions of losses into loss-derivatives, thereby bypassing the need for explicit loss functions. We theoretically establish the statistical consistency of our approach and provide insights into its benefits.
  arXiv  Detail & Related papers  (2024-09-02T02:40:42Z)
• Rejection via Learning Density Ratios [50.91522897152437]
  Classification with rejection emerges as a learning paradigm which allows models to abstain from making predictions.<n>We propose a different distributional perspective, where we seek to find an idealized data distribution which maximizes a pretrained model's performance.<n>Our framework is tested empirically over clean and noisy datasets.
  arXiv  Detail & Related papers  (2024-05-29T01:32:17Z)
• Robust T-Loss for Medical Image Segmentation [56.524774292536264]
  This paper presents a new robust loss function, the T-Loss, for medical image segmentation. The proposed loss is based on the negative log-likelihood of the Student-t distribution and can effectively handle outliers in the data. Our experiments show that the T-Loss outperforms traditional loss functions in terms of dice scores on two public medical datasets.
  arXiv  Detail & Related papers  (2023-06-01T14:49:40Z)
• Cross-Entropy Loss Functions: Theoretical Analysis and Applications [27.3569897539488]
  We present a theoretical analysis of a broad family of loss functions, that includes cross-entropy (or logistic loss), generalized cross-entropy, the mean absolute error and other cross-entropy-like loss functions. We show that these loss functions are beneficial in the adversarial setting by proving that they admit $H$-consistency bounds. This leads to new adversarial robustness algorithms that consist of minimizing a regularized smooth adversarial comp-sum loss.
  arXiv  Detail & Related papers  (2023-04-14T17:58:23Z)
• What can we learn about a generated image corrupting its latent representation? [57.1841740328509]
  We investigate the hypothesis that we can predict image quality based on its latent representation in the GANs bottleneck. We achieve this by corrupting the latent representation with noise and generating multiple outputs.
  arXiv  Detail & Related papers  (2022-10-12T14:40:32Z)
• Learning with Noisy Labels via Sparse Regularization [76.31104997491695]
  Learning with noisy labels is an important task for training accurate deep neural networks. Some commonly-used loss functions, such as Cross Entropy (CE), suffer from severe overfitting to noisy labels. We introduce the sparse regularization strategy to approximate the one-hot constraint.
  arXiv  Detail & Related papers  (2021-07-31T09:40:23Z)
• Lower-bounded proper losses for weakly supervised classification [73.974163801142]
  We discuss the problem of weakly supervised learning of classification, in which instances are given weak labels. We derive a representation theorem for proper losses in supervised learning, which dualizes the Savage representation. We experimentally demonstrate the effectiveness of our proposed approach, as compared to improper or unbounded losses.
  arXiv  Detail & Related papers  (2021-03-04T08:47:07Z)
• All your loss are belong to Bayes [28.393499629583786]
  Loss functions are a cornerstone of machine learning and the starting point of most algorithms. We introduce a trick on squared Gaussian Processes to obtain a random process whose paths are compliant source functions. Experimental results demonstrate substantial improvements over the state of the art.
  arXiv  Detail & Related papers  (2020-06-08T14:31:21Z)
• Supervised Learning: No Loss No Cry [51.07683542418145]
  Supervised learning requires the specification of a loss function to minimise. This paper revisits the sc SLIsotron algorithm of Kakade et al. (2011) through a novel lens. We show how it provides a principled procedure for learning the loss.
  arXiv  Detail & Related papers  (2020-02-10T05:30:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.