Supervised Learning with General Risk Functionals
- URL: http://arxiv.org/abs/2206.13648v1
- Date: Mon, 27 Jun 2022 22:11:05 GMT
- Title: Supervised Learning with General Risk Functionals
- Authors: Liu Leqi, Audrey Huang, Zachary C. Lipton, Kamyar Azizzadenesheli
- Abstract summary: Standard uniform convergence results bound the generalization gap of the expected loss over a hypothesis class.
We establish the first uniform convergence results for estimating the CDF of the loss distribution, yielding guarantees that hold simultaneously both over all H"older risk functionals and over all hypotheses.
- Score: 28.918233583859134
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Standard uniform convergence results bound the generalization gap of the
expected loss over a hypothesis class. The emergence of risk-sensitive learning
requires generalization guarantees for functionals of the loss distribution
beyond the expectation. While prior works specialize in uniform convergence of
particular functionals, our work provides uniform convergence for a general
class of H\"older risk functionals for which the closeness in the Cumulative
Distribution Function (CDF) entails closeness in risk. We establish the first
uniform convergence results for estimating the CDF of the loss distribution,
yielding guarantees that hold simultaneously both over all H\"older risk
functionals and over all hypotheses. Thus licensed to perform empirical risk
minimization, we develop practical gradient-based methods for minimizing
distortion risks (widely studied subset of H\"older risks that subsumes the
spectral risks, including the mean, conditional value at risk, cumulative
prospect theory risks, and others) and provide convergence guarantees. In
experiments, we demonstrate the efficacy of our learning procedure, both in
settings where uniform convergence results hold and in high-dimensional
settings with deep networks.
Related papers
- Provable Risk-Sensitive Distributional Reinforcement Learning with
General Function Approximation [54.61816424792866]
We introduce a general framework on Risk-Sensitive Distributional Reinforcement Learning (RS-DisRL), with static Lipschitz Risk Measures (LRM) and general function approximation.
We design two innovative meta-algorithms: textttRS-DisRL-M, a model-based strategy for model-based function approximation, and textttRS-DisRL-V, a model-free approach for general value function approximation.
arXiv Detail & Related papers (2024-02-28T08:43:18Z) - Strong Duality Relations in Nonconvex Risk-Constrained Learning [0.0]
J. J. Uhl's convexity for general infinite dimensional Banach spaces is an extension of A. A. Lynovs.
We show that constrained classification and regression can be treated under a unifying lens, while certain restrictive assumptions enforced in the current literature are a new state of the art.
arXiv Detail & Related papers (2023-12-02T11:21:00Z) - Pitfall of Optimism: Distributional Reinforcement Learning by
Randomizing Risk Criterion [9.35556128467037]
We present a novel distributional reinforcement learning algorithm that selects actions by randomizing risk criterion to avoid one-sided tendency on risk.
Our theoretical results support that the proposed method does not fall into biased exploration and is guaranteed to converge to an optimal return.
arXiv Detail & Related papers (2023-10-25T10:53:04Z) - Non-Asymptotic Bounds for Adversarial Excess Risk under Misspecified
Models [9.65010022854885]
We show that adversarial risk is equivalent to the risk induced by a distributional adversarial attack under certain smoothness conditions.
To evaluate the generalization performance of the adversarial estimator, we study the adversarial excess risk.
arXiv Detail & Related papers (2023-09-02T00:51:19Z) - Domain Generalization without Excess Empirical Risk [83.26052467843725]
A common approach is designing a data-driven surrogate penalty to capture generalization and minimize the empirical risk jointly with the penalty.
We argue that a significant failure mode of this recipe is an excess risk due to an erroneous penalty or hardness in joint optimization.
We present an approach that eliminates this problem. Instead of jointly minimizing empirical risk with the penalty, we minimize the penalty under the constraint of optimality of the empirical risk.
arXiv Detail & Related papers (2023-08-30T08:46:46Z) - Extreme Risk Mitigation in Reinforcement Learning using Extreme Value
Theory [10.288413564829579]
A critical aspect of risk awareness involves modeling highly rare risk events (rewards) that could potentially lead to catastrophic outcomes.
While risk-aware RL techniques do exist, their level of risk aversion heavily relies on the precision of the state-action value function estimation.
Our work proposes to enhance the resilience of RL agents when faced with very rare and risky events by focusing on refining the predictions of the extreme values predicted by the state-action value function distribution.
arXiv Detail & Related papers (2023-08-24T18:23:59Z) - A Generalized Unbiased Risk Estimator for Learning with Augmented
Classes [70.20752731393938]
Given unlabeled data, an unbiased risk estimator (URE) can be derived, which can be minimized for LAC with theoretical guarantees.
We propose a generalized URE that can be equipped with arbitrary loss functions while maintaining the theoretical guarantees.
arXiv Detail & Related papers (2023-06-12T06:52:04Z) - Safe Deployment for Counterfactual Learning to Rank with Exposure-Based
Risk Minimization [63.93275508300137]
We introduce a novel risk-aware Counterfactual Learning To Rank method with theoretical guarantees for safe deployment.
Our experimental results demonstrate the efficacy of our proposed method, which is effective at avoiding initial periods of bad performance when little data is available.
arXiv Detail & Related papers (2023-04-26T15:54:23Z) - Risk-Sensitive Deep RL: Variance-Constrained Actor-Critic Provably Finds
Globally Optimal Policy [95.98698822755227]
We make the first attempt to study risk-sensitive deep reinforcement learning under the average reward setting with the variance risk criteria.
We propose an actor-critic algorithm that iteratively and efficiently updates the policy, the Lagrange multiplier, and the Fenchel dual variable.
arXiv Detail & Related papers (2020-12-28T05:02:26Z) - Learning Bounds for Risk-sensitive Learning [86.50262971918276]
In risk-sensitive learning, one aims to find a hypothesis that minimizes a risk-averse (or risk-seeking) measure of loss.
We study the generalization properties of risk-sensitive learning schemes whose optimand is described via optimized certainty equivalents.
arXiv Detail & Related papers (2020-06-15T05:25:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.