Two steps to risk sensitivity

• URL: http://arxiv.org/abs/2111.06803v1
• Date: Fri, 12 Nov 2021 16:27:47 GMT
• Title: Two steps to risk sensitivity
• Authors: Chris Gagne and Peter Dayan
• Abstract summary: conditional value-at-risk (CVaR) is a risk measure for modeling human and animal planning. We adopt a conventional distributional approach to CVaR in a sequential setting and reanalyze the choices of human decision-makers. We then consider a further critical property of risk sensitivity, namely time consistency, showing alternatives to this form of CVaR.
• Score: 4.974890682815778
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Distributional reinforcement learning (RL) -- in which agents learn about all the possible long-term consequences of their actions, and not just the expected value -- is of great recent interest. One of the most important affordances of a distributional view is facilitating a modern, measured, approach to risk when outcomes are not completely certain. By contrast, psychological and neuroscientific investigations into decision making under risk have utilized a variety of more venerable theoretical models such as prospect theory that lack axiomatically desirable properties such as coherence. Here, we consider a particularly relevant risk measure for modeling human and animal planning, called conditional value-at-risk (CVaR), which quantifies worst-case outcomes (e.g., vehicle accidents or predation). We first adopt a conventional distributional approach to CVaR in a sequential setting and reanalyze the choices of human decision-makers in the well-known two-step task, revealing substantial risk aversion that had been lurking under stickiness and perseveration. We then consider a further critical property of risk sensitivity, namely time consistency, showing alternatives to this form of CVaR that enjoy this desirable characteristic. We use simulations to examine settings in which the various forms differ in ways that have implications for human and animal planning and behavior.

Related papers

• Data-driven decision-making under uncertainty with entropic risk measure [5.407319151576265]
  The entropic risk measure is widely used in high-stakes decision making to account for tail risks associated with an uncertain loss. To debias the empirical entropic risk estimator, we propose a strongly consistent bootstrapping procedure. We show that cross validation methods can result in significantly higher out-of-sample risk for the insurer if the bias in validation performance is not corrected for.
  arXiv  Detail & Related papers  (2024-09-30T04:02:52Z)
• Controlling Risk of Retrieval-augmented Generation: A Counterfactual Prompting Framework [77.45983464131977]
  We focus on how likely it is that a RAG model's prediction is incorrect, resulting in uncontrollable risks in real-world applications. Our research identifies two critical latent factors affecting RAG's confidence in its predictions. We develop a counterfactual prompting framework that induces the models to alter these factors and analyzes the effect on their answers.
  arXiv  Detail & Related papers  (2024-09-24T14:52:14Z)
• Extreme Risk Mitigation in Reinforcement Learning using Extreme Value Theory [10.288413564829579]
  A critical aspect of risk awareness involves modeling highly rare risk events (rewards) that could potentially lead to catastrophic outcomes. While risk-aware RL techniques do exist, their level of risk aversion heavily relies on the precision of the state-action value function estimation. Our work proposes to enhance the resilience of RL agents when faced with very rare and risky events by focusing on refining the predictions of the extreme values predicted by the state-action value function distribution.
  arXiv  Detail & Related papers  (2023-08-24T18:23:59Z)
• On (assessing) the fairness of risk score models [2.0646127669654826]
  Risk models are of interest for a number of reasons, including the fact that they communicate uncertainty about the potential outcomes to users. We identify the provision of similar value to different groups as a key desideratum for risk score fairness. We introduce a novel calibration error metric that is less sample size-biased than previously proposed metrics.
  arXiv  Detail & Related papers  (2023-02-17T12:45:51Z)
• Mitigating multiple descents: A model-agnostic framework for risk monotonization [84.6382406922369]
  We develop a general framework for risk monotonization based on cross-validation. We propose two data-driven methodologies, namely zero- and one-step, that are akin to bagging and boosting.
  arXiv  Detail & Related papers  (2022-05-25T17:41:40Z)
• A Survey of Risk-Aware Multi-Armed Bandits [84.67376599822569]
  We review various risk measures of interest, and comment on their properties. We consider algorithms for the regret minimization setting, where the exploration-exploitation trade-off manifests. We conclude by commenting on persisting challenges and fertile areas for future research.
  arXiv  Detail & Related papers  (2022-05-12T02:20:34Z)
• Empirical Estimates on Hand Manipulation are Recoverable: A Step Towards Individualized and Explainable Robotic Support in Everyday Activities [80.37857025201036]
  Key challenge for robotic systems is to figure out the behavior of another agent. Processing correct inferences is especially challenging when (confounding) factors are not controlled experimentally. We propose equipping robots with the necessary tools to conduct observational studies on people.
  arXiv  Detail & Related papers  (2022-01-27T22:15:56Z)
• Catastrophe, Compounding & Consistency in Choice [4.974890682815778]
  Conditional value-at-risk (CVaR) precisely characterizes the influence that rare, catastrophic events can exert over decisions. These examples can ground future experiments with the broader aim of characterizing risk attitudes.
  arXiv  Detail & Related papers  (2021-11-12T16:33:06Z)
• Enabling risk-aware Reinforcement Learning for medical interventions through uncertainty decomposition [9.208828373290487]
  Reinforcement Learning (RL) is emerging as tool for tackling complex control and decision-making problems. It is often challenging to bridge the gap between an apparently optimal policy learnt by an agent and its real-world deployment. Here we propose how a distributional approach (UA-DQN) can be recast to render uncertainties by decomposing the net effects of each uncertainty.
  arXiv  Detail & Related papers  (2021-09-16T09:36:53Z)
• Learning Bounds for Risk-sensitive Learning [86.50262971918276]
  In risk-sensitive learning, one aims to find a hypothesis that minimizes a risk-averse (or risk-seeking) measure of loss. We study the generalization properties of risk-sensitive learning schemes whose optimand is described via optimized certainty equivalents.
  arXiv  Detail & Related papers  (2020-06-15T05:25:02Z)
• A General Framework for Survival Analysis and Multi-State Modelling [70.31153478610229]
  We use neural ordinary differential equations as a flexible and general method for estimating multi-state survival models. We show that our model exhibits state-of-the-art performance on popular survival data sets and demonstrate its efficacy in a multi-state setting.
  arXiv  Detail & Related papers  (2020-06-08T19:24:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.