Risk-Controlling Model Selection via Guided Bayesian Optimization
- URL: http://arxiv.org/abs/2312.01692v1
- Date: Mon, 4 Dec 2023 07:29:44 GMT
- Title: Risk-Controlling Model Selection via Guided Bayesian Optimization
- Authors: Bracha Laufer-Goldshtein, Adam Fisch, Regina Barzilay, Tommi Jaakkola
- Abstract summary: We find a configuration that adheres to user-specified limits on certain risks while being useful with respect to other conflicting metrics.
Our method identifies a set of optimal configurations residing in a designated region of interest.
We demonstrate the effectiveness of our approach on a range of tasks with multiple desiderata, including low error rates, equitable predictions, handling spurious correlations, managing rate and distortion in generative models, and reducing computational costs.
- Score: 35.53469358591976
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Adjustable hyperparameters of machine learning models typically impact
various key trade-offs such as accuracy, fairness, robustness, or inference
cost. Our goal in this paper is to find a configuration that adheres to
user-specified limits on certain risks while being useful with respect to other
conflicting metrics. We solve this by combining Bayesian Optimization (BO) with
rigorous risk-controlling procedures, where our core idea is to steer BO
towards an efficient testing strategy. Our BO method identifies a set of Pareto
optimal configurations residing in a designated region of interest. The
resulting candidates are statistically verified and the best-performing
configuration is selected with guaranteed risk levels. We demonstrate the
effectiveness of our approach on a range of tasks with multiple desiderata,
including low error rates, equitable predictions, handling spurious
correlations, managing rate and distortion in generative models, and reducing
computational costs.
Related papers
- Quantile Learn-Then-Test: Quantile-Based Risk Control for Hyperparameter Optimization [36.14499894307206]
This work introduces a variant of learn-then-test (LTT) that is designed to provide statistical guarantees on quantiles of a risk measure.
We illustrate the practical advantages of this approach by applying the proposed algorithm to a radio access scheduling problem.
arXiv Detail & Related papers (2024-07-24T15:30:12Z) - Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer [52.09480867526656]
We identify the source of misalignment as a form of distributional shift and uncertainty in learning human preferences.
To mitigate overoptimization, we first propose a theoretical algorithm that chooses the best policy for an adversarially chosen reward model.
Using the equivalence between reward models and the corresponding optimal policy, the algorithm features a simple objective that combines a preference optimization loss and a supervised learning loss.
arXiv Detail & Related papers (2024-05-26T05:38:50Z) - Likelihood Ratio Confidence Sets for Sequential Decision Making [51.66638486226482]
We revisit the likelihood-based inference principle and propose to use likelihood ratios to construct valid confidence sequences.
Our method is especially suitable for problems with well-specified likelihoods.
We show how to provably choose the best sequence of estimators and shed light on connections to online convex optimization.
arXiv Detail & Related papers (2023-11-08T00:10:21Z) - When Demonstrations Meet Generative World Models: A Maximum Likelihood
Framework for Offline Inverse Reinforcement Learning [62.00672284480755]
This paper aims to recover the structure of rewards and environment dynamics that underlie observed actions in a fixed, finite set of demonstrations from an expert agent.
Accurate models of expertise in executing a task has applications in safety-sensitive applications such as clinical decision making and autonomous driving.
arXiv Detail & Related papers (2023-02-15T04:14:20Z) - Efficiently Controlling Multiple Risks with Pareto Testing [34.83506056862348]
We propose a two-stage process which combines multi-objective optimization with multiple hypothesis testing.
We demonstrate the effectiveness of our approach to reliably accelerate the execution of large-scale Transformer models in natural language processing (NLP) applications.
arXiv Detail & Related papers (2022-10-14T15:54:39Z) - Error-based Knockoffs Inference for Controlled Feature Selection [49.99321384855201]
We propose an error-based knockoff inference method by integrating the knockoff features, the error-based feature importance statistics, and the stepdown procedure together.
The proposed inference procedure does not require specifying a regression model and can handle feature selection with theoretical guarantees.
arXiv Detail & Related papers (2022-03-09T01:55:59Z) - Learning Robust Output Control Barrier Functions from Safe Expert Demonstrations [50.37808220291108]
This paper addresses learning safe output feedback control laws from partial observations of expert demonstrations.
We first propose robust output control barrier functions (ROCBFs) as a means to guarantee safety.
We then formulate an optimization problem to learn ROCBFs from expert demonstrations that exhibit safe system behavior.
arXiv Detail & Related papers (2021-11-18T23:21:00Z) - Risk-averse Heteroscedastic Bayesian Optimization [45.12421486836736]
We propose a novel risk-averse heteroscedastic Bayesian optimization algorithm (RAHBO)
RAHBO aims to identify a solution with high return and low noise variance, while learning the noise distribution on the fly.
We provide a robust rule to report the final decision point for applications where only a single solution must be identified.
arXiv Detail & Related papers (2021-11-05T17:38:34Z) - Modeling the Second Player in Distributionally Robust Optimization [90.25995710696425]
We argue for the use of neural generative models to characterize the worst-case distribution.
This approach poses a number of implementation and optimization challenges.
We find that the proposed approach yields models that are more robust than comparable baselines.
arXiv Detail & Related papers (2021-03-18T14:26:26Z) - Continuous Mean-Covariance Bandits [39.820490484375156]
We propose a novel Continuous Mean-Covariance Bandit model to take into account option correlation.
In CMCB, there is a learner who sequentially chooses weight vectors on given options and observes random feedback according to the decisions.
We propose novel algorithms with optimal regrets (within logarithmic factors) and provide matching lower bounds to validate their optimalities.
arXiv Detail & Related papers (2021-02-24T06:37:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.