Learning Bounds for Risk-sensitive Learning
- URL: http://arxiv.org/abs/2006.08138v2
- Date: Mon, 4 Jan 2021 07:14:49 GMT
- Title: Learning Bounds for Risk-sensitive Learning
- Authors: Jaeho Lee, Sejun Park, Jinwoo Shin
- Abstract summary: In risk-sensitive learning, one aims to find a hypothesis that minimizes a risk-averse (or risk-seeking) measure of loss.
We study the generalization properties of risk-sensitive learning schemes whose optimand is described via optimized certainty equivalents.
- Score: 86.50262971918276
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In risk-sensitive learning, one aims to find a hypothesis that minimizes a
risk-averse (or risk-seeking) measure of loss, instead of the standard expected
loss. In this paper, we propose to study the generalization properties of
risk-sensitive learning schemes whose optimand is described via optimized
certainty equivalents (OCE): our general scheme can handle various known risks,
e.g., the entropic risk, mean-variance, and conditional value-at-risk, as
special cases. We provide two learning bounds on the performance of empirical
OCE minimizer. The first result gives an OCE guarantee based on the Rademacher
average of the hypothesis space, which generalizes and improves existing
results on the expected loss and the conditional value-at-risk. The second
result, based on a novel variance-based characterization of OCE, gives an
expected loss guarantee with a suppressed dependence on the smoothness of the
selected OCE. Finally, we demonstrate the practical implications of the
proposed bounds via exploratory experiments on neural networks.
Related papers
- Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learning [19.292214425524303]
We study risk-sensitive reinforcement learning (RL), a crucial field due to its ability to enhance decision-making in scenarios where it is essential to manage uncertainty and minimize potential adverse outcomes.
Our work focuses on applying the entropic risk measure to RL problems.
We center on the linear Markov Decision Process (MDP) setting, a well-regarded theoretical framework that has yet to be examined from a risk-sensitive standpoint.
arXiv Detail & Related papers (2024-07-10T13:09:52Z) - Model-Based Epistemic Variance of Values for Risk-Aware Policy Optimization [59.758009422067]
We consider the problem of quantifying uncertainty over expected cumulative rewards in model-based reinforcement learning.
We propose a new uncertainty Bellman equation (UBE) whose solution converges to the true posterior variance over values.
We introduce a general-purpose policy optimization algorithm, Q-Uncertainty Soft Actor-Critic (QU-SAC) that can be applied for either risk-seeking or risk-averse policy optimization.
arXiv Detail & Related papers (2023-12-07T15:55:58Z) - Strong Duality Relations in Nonconvex Risk-Constrained Learning [0.0]
J. J. Uhl's convexity for general infinite dimensional Banach spaces is an extension of A. A. Lynovs.
We show that constrained classification and regression can be treated under a unifying lens, while certain restrictive assumptions enforced in the current literature are a new state of the art.
arXiv Detail & Related papers (2023-12-02T11:21:00Z) - Domain Generalization without Excess Empirical Risk [83.26052467843725]
A common approach is designing a data-driven surrogate penalty to capture generalization and minimize the empirical risk jointly with the penalty.
We argue that a significant failure mode of this recipe is an excess risk due to an erroneous penalty or hardness in joint optimization.
We present an approach that eliminates this problem. Instead of jointly minimizing empirical risk with the penalty, we minimize the penalty under the constraint of optimality of the empirical risk.
arXiv Detail & Related papers (2023-08-30T08:46:46Z) - A Distribution Optimization Framework for Confidence Bounds of Risk
Measures [23.46659319363579]
We present a distribution optimization framework that significantly improves confidence bounds for various risk measures compared to previous methods.
Our framework encompasses popular risk measures such as the entropic risk measure, conditional value at risk (CVaR), spectral risk measure, distortion risk measure, equivalent certainty, and rank-dependent expected utility.
arXiv Detail & Related papers (2023-06-12T12:13:06Z) - A Generalized Unbiased Risk Estimator for Learning with Augmented
Classes [70.20752731393938]
Given unlabeled data, an unbiased risk estimator (URE) can be derived, which can be minimized for LAC with theoretical guarantees.
We propose a generalized URE that can be equipped with arbitrary loss functions while maintaining the theoretical guarantees.
arXiv Detail & Related papers (2023-06-12T06:52:04Z) - Deep Learning for Systemic Risk Measures [3.274367403737527]
The aim of this paper is to study a new methodological framework for systemic risk measures.
Under this new framework, systemic risk measures can be interpreted as the minimal amount of cash that secures the aggregated system.
Deep learning is increasingly receiving attention in financial modelings and risk management.
arXiv Detail & Related papers (2022-07-02T05:01:19Z) - Supervised Learning with General Risk Functionals [28.918233583859134]
Standard uniform convergence results bound the generalization gap of the expected loss over a hypothesis class.
We establish the first uniform convergence results for estimating the CDF of the loss distribution, yielding guarantees that hold simultaneously both over all H"older risk functionals and over all hypotheses.
arXiv Detail & Related papers (2022-06-27T22:11:05Z) - Efficient Risk-Averse Reinforcement Learning [79.61412643761034]
In risk-averse reinforcement learning (RL), the goal is to optimize some risk measure of the returns.
We prove that under certain conditions this inevitably leads to a local-optimum barrier, and propose a soft risk mechanism to bypass it.
We demonstrate improved risk aversion in maze navigation, autonomous driving, and resource allocation benchmarks.
arXiv Detail & Related papers (2022-05-10T19:40:52Z) - Risk-Sensitive Deep RL: Variance-Constrained Actor-Critic Provably Finds
Globally Optimal Policy [95.98698822755227]
We make the first attempt to study risk-sensitive deep reinforcement learning under the average reward setting with the variance risk criteria.
We propose an actor-critic algorithm that iteratively and efficiently updates the policy, the Lagrange multiplier, and the Fenchel dual variable.
arXiv Detail & Related papers (2020-12-28T05:02:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.