Learning with risks based on M-location

• URL: http://arxiv.org/abs/2012.02424v2
• Date: Mon, 26 Apr 2021 00:37:11 GMT
• Title: Learning with risks based on M-location
• Authors: Matthew J. Holland
• Abstract summary: We study a new class of risks defined in terms of the location and deviation of the loss distribution. The class is easily implemented as a wrapper around any smooth loss.
• Score: 6.903929927172917
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we study a new class of risks defined in terms of the location and deviation of the loss distribution, generalizing far beyond classical mean-variance risk functions. The class is easily implemented as a wrapper around any smooth loss, it admits finite-sample stationarity guarantees for stochastic gradient methods, it is straightforward to interpret and adjust, with close links to M-estimators of the loss location, and has a salient effect on the test loss distribution.

Related papers

• Understanding Transfer Learning via Mean-field Analysis [5.7150083558242075]
  We consider two main transfer learning scenarios, $alpha$-ERM and fine-tuning with the KL-regularized empirical risk minimization. We show the benefits of transfer learning with a one-hidden-layer neural network in the mean-field regime.
  arXiv  Detail & Related papers  (2024-10-22T16:00:44Z)
• An Alternative to Variance: Gini Deviation for Risk-averse Policy Gradient [35.01235012813407]
  Restricting the variance of a policy's return is a popular choice in risk-averse Reinforcement Learning. Recent methods restrict the per-step reward variance as a proxy. We propose to use an alternative risk measure, Gini deviation, as a substitute.
  arXiv  Detail & Related papers  (2023-07-17T22:08:27Z)
• A Generalized Unbiased Risk Estimator for Learning with Augmented Classes [70.20752731393938]
  Given unlabeled data, an unbiased risk estimator (URE) can be derived, which can be minimized for LAC with theoretical guarantees. We propose a generalized URE that can be equipped with arbitrary loss functions while maintaining the theoretical guarantees.
  arXiv  Detail & Related papers  (2023-06-12T06:52:04Z)
• On the Variance, Admissibility, and Stability of Empirical Risk Minimization [80.26309576810844]
  Empirical Risk Minimization (ERM) with squared loss may attain minimax suboptimal error rates. We show that under mild assumptions, the suboptimality of ERM must be due to large bias rather than variance. We also show that our estimates imply stability of ERM, complementing the main result of Caponnetto and Rakhlin (2006) for non-Donsker classes.
  arXiv  Detail & Related papers  (2023-05-29T15:25:48Z)
• Tailoring to the Tails: Risk Measures for Fine-Grained Tail Sensitivity [10.482805367361818]
  Expected risk rearrangement (ERM) is at the core of machine learning systems. We propose a general approach to construct risk measures which exhibit a desired tail sensitivity.
  arXiv  Detail & Related papers  (2022-08-05T09:51:18Z)
• Supervised Learning with General Risk Functionals [28.918233583859134]
  Standard uniform convergence results bound the generalization gap of the expected loss over a hypothesis class. We establish the first uniform convergence results for estimating the CDF of the loss distribution, yielding guarantees that hold simultaneously both over all H"older risk functionals and over all hypotheses.
  arXiv  Detail & Related papers  (2022-06-27T22:11:05Z)
• Risk Minimization from Adaptively Collected Data: Guarantees for Supervised and Policy Learning [57.88785630755165]
  Empirical risk minimization (ERM) is the workhorse of machine learning, but its model-agnostic guarantees can fail when we use adaptively collected data. We study a generic importance sampling weighted ERM algorithm for using adaptively collected data to minimize the average of a loss function over a hypothesis class. For policy learning, we provide rate-optimal regret guarantees that close an open gap in the existing literature whenever exploration decays to zero.
  arXiv  Detail & Related papers  (2021-06-03T09:50:13Z)
• Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation [109.73983088432364]
  We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA) We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
  arXiv  Detail & Related papers  (2020-10-09T15:42:35Z)
• Unbiased Risk Estimators Can Mislead: A Case Study of Learning with Complementary Labels [92.98756432746482]
  We study a weakly supervised problem called learning with complementary labels. We show that the quality of gradient estimation matters more in risk minimization. We propose a novel surrogate complementary loss(SCL) framework that trades zero bias with reduced variance.
  arXiv  Detail & Related papers  (2020-07-05T04:19:37Z)
• Learning Bounds for Risk-sensitive Learning [86.50262971918276]
  In risk-sensitive learning, one aims to find a hypothesis that minimizes a risk-averse (or risk-seeking) measure of loss. We study the generalization properties of risk-sensitive learning schemes whose optimand is described via optimized certainty equivalents.
  arXiv  Detail & Related papers  (2020-06-15T05:25:02Z)
• Adversarial Classification via Distributional Robustness with Wasserstein Ambiguity [12.576828231302134]
  Under Wasserstein ambiguity, the model aims to minimize the value-at-risk of misclassification. We show that, despite the non-marginity of this classification, standard descent methods appear to converger for this problem.
  arXiv  Detail & Related papers  (2020-05-28T07:28:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.