Robust Losses for Decision-Focused Learning
- URL: http://arxiv.org/abs/2310.04328v1
- Date: Fri, 6 Oct 2023 15:45:10 GMT
- Title: Robust Losses for Decision-Focused Learning
- Authors: Noah Schutte, Krzysztof Postek, Neil Yorke-Smith
- Abstract summary: Decision-focused learning approaches are proposed to minimize regret in suboptimal decisions.
In this paper, we evaluate the effect of aleatoric uncertainty on the accuracy of empirical regret as a surrogate.
- Score: 3.3326409357902245
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Optimization models used to make discrete decisions often contain uncertain
parameters that are context-dependent and are estimated through prediction. To
account for the quality of the decision made based on the prediction,
decision-focused learning (end-to-end predict-then-optimize) aims at training
the predictive model to minimize regret, i.e., the loss incurred by making a
suboptimal decision. Despite the challenge of this loss function being possibly
non-convex and in general non-differentiable, effective gradient-based learning
approaches have been proposed to minimize the expected loss, using the
empirical loss as a surrogate. However, empirical regret can be an ineffective
surrogate because the uncertainty in the optimization model makes the empirical
regret unequal to the expected regret in expectation. To illustrate the impact
of this inequality, we evaluate the effect of aleatoric and epistemic
uncertainty on the accuracy of empirical regret as a surrogate. Next, we
propose three robust loss functions that more closely approximate expected
regret. Experimental results show that training two state-of-the-art
decision-focused learning approaches using robust regret losses improves
test-sample empirical regret in general while keeping computational time
equivalent relative to the number of training epochs.
Related papers
- Asymptotically Optimal Regret for Black-Box Predict-then-Optimize [7.412445894287709]
We study new black-box predict-then-optimize problems that lack special structure and where we only observe the reward from the action taken.
We present a novel loss function, which we call Empirical Soft Regret (ESR), designed to significantly improve reward when used in training.
We also show our approach significantly outperforms state-of-the-art algorithms on real-world decision-making problems in news recommendation and personalized healthcare.
arXiv Detail & Related papers (2024-06-12T04:46:23Z) - Optimistic Regret Bounds for Online Learning in Adversarial Markov Decision Processes [5.116582735311639]
We introduce and study a new variant of AMDP, which aims to minimize regret while utilizing a set of cost predictors.
We develop a new policy search method that achieves a sublinear optimistic regret with high probability, that is a regret bound which gracefully degrades with the estimation power of the cost predictors.
arXiv Detail & Related papers (2024-05-03T15:44:31Z) - Model-Based Uncertainty in Value Functions [89.31922008981735]
We focus on characterizing the variance over values induced by a distribution over MDPs.
Previous work upper bounds the posterior variance over values by solving a so-called uncertainty Bellman equation.
We propose a new uncertainty Bellman equation whose solution converges to the true posterior variance over values.
arXiv Detail & Related papers (2023-02-24T09:18:27Z) - On the Difficulty of Epistemic Uncertainty Quantification in Machine
Learning: The Case of Direct Uncertainty Estimation through Loss Minimisation [8.298716599039501]
Uncertainty quantification has received increasing attention in machine learning.
The latter refers to the learner's (lack of) knowledge and appears to be especially difficult to measure and quantify.
We show that loss minimisation does not work for second-order predictors.
arXiv Detail & Related papers (2022-03-11T17:26:05Z) - Improving MC-Dropout Uncertainty Estimates with Calibration Error-based
Optimization [18.22429945073576]
We propose two new loss functions by combining cross entropy with Expected Error (ECE) and Predictive Entropy (PE)
Our results confirmed the great impact of the new hybrid loss functions for minimising the overlap between the distributions of uncertainty estimates for correct and incorrect predictions without sacrificing the model's overall performance.
arXiv Detail & Related papers (2021-10-07T08:31:23Z) - A Regret Minimization Approach to Iterative Learning Control [61.37088759497583]
We propose a new performance metric, planning regret, which replaces the standard uncertainty assumptions with worst case regret.
We provide theoretical and empirical evidence that the proposed algorithm outperforms existing methods on several benchmarks.
arXiv Detail & Related papers (2021-02-26T13:48:49Z) - DEUP: Direct Epistemic Uncertainty Prediction [56.087230230128185]
Epistemic uncertainty is part of out-of-sample prediction error due to the lack of knowledge of the learner.
We propose a principled approach for directly estimating epistemic uncertainty by learning to predict generalization error and subtracting an estimate of aleatoric uncertainty.
arXiv Detail & Related papers (2021-02-16T23:50:35Z) - The Aleatoric Uncertainty Estimation Using a Separate Formulation with
Virtual Residuals [51.71066839337174]
Existing methods can quantify the error in the target estimation, but they tend to underestimate it.
We propose a new separable formulation for the estimation of a signal and of its uncertainty, avoiding the effect of overfitting.
We demonstrate that the proposed method outperforms a state-of-the-art technique for signal and uncertainty estimation.
arXiv Detail & Related papers (2020-11-03T12:11:27Z) - On the Loss Landscape of Adversarial Training: Identifying Challenges
and How to Overcome Them [57.957466608543676]
We analyze the influence of adversarial training on the loss landscape of machine learning models.
We show that the adversarial loss landscape is less favorable to optimization, due to increased curvature and more scattered gradients.
arXiv Detail & Related papers (2020-06-15T13:50:23Z) - Learning to Predict Error for MRI Reconstruction [67.76632988696943]
We demonstrate that predictive uncertainty estimated by the current methods does not highly correlate with prediction error.
We propose a novel method that estimates the target labels and magnitude of the prediction error in two steps.
arXiv Detail & Related papers (2020-02-13T15:55:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.