Meta-Learning Hypothesis Spaces for Sequential Decision-making
- URL: http://arxiv.org/abs/2202.00602v1
- Date: Tue, 1 Feb 2022 17:46:51 GMT
- Title: Meta-Learning Hypothesis Spaces for Sequential Decision-making
- Authors: Parnian Kassraie, Jonas Rothfuss, Andreas Krause
- Abstract summary: We propose to meta-learn a kernel from offline data (Meta-KeL)
Under mild conditions, we guarantee that our estimated RKHS yields valid confidence sets.
We also empirically evaluate the effectiveness of our approach on a Bayesian optimization task.
- Score: 79.73213540203389
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Obtaining reliable, adaptive confidence sets for prediction functions
(hypotheses) is a central challenge in sequential decision-making tasks, such
as bandits and model-based reinforcement learning. These confidence sets
typically rely on prior assumptions on the hypothesis space, e.g., the known
kernel of a Reproducing Kernel Hilbert Space (RKHS). Hand-designing such
kernels is error prone, and misspecification may lead to poor or unsafe
performance. In this work, we propose to meta-learn a kernel from offline data
(Meta-KeL). For the case where the unknown kernel is a combination of known
base kernels, we develop an estimator based on structured sparsity. Under mild
conditions, we guarantee that our estimated RKHS yields valid confidence sets
that, with increasing amounts of offline data, become as tight as those given
the true unknown kernel. We demonstrate our approach on the kernelized bandit
problem (a.k.a.~Bayesian optimization), where we establish regret bounds
competitive with those given the true kernel. We also empirically evaluate the
effectiveness of our approach on a Bayesian optimization task.
Related papers
- Kernel-Based Function Approximation for Average Reward Reinforcement Learning: An Optimist No-Regret Algorithm [11.024396385514864]
We consider kernel-based function for approximation RL in the infinite horizon average reward setting.
We propose an optimistic algorithm, similar to acquisition function based algorithms in the special case of bandits.
arXiv Detail & Related papers (2024-10-30T23:04:10Z) - Tighter Confidence Bounds for Sequential Kernel Regression [3.683202928838613]
We use martingale tail inequalities to establish new confidence bounds for sequential kernel regression.
Our confidence bounds can be computed by solving a conic program, although this bare version quickly becomes impractical.
We find that when our confidence bounds replace existing ones, the KernelUCB algorithm has better empirical performance, a matching worst-case performance guarantee and comparable computational cost.
arXiv Detail & Related papers (2024-03-19T13:47:35Z) - Information-Theoretic Safe Bayesian Optimization [59.758009422067005]
We consider a sequential decision making task, where the goal is to optimize an unknown function without evaluating parameters that violate an unknown (safety) constraint.
Most current methods rely on a discretization of the domain and cannot be directly extended to the continuous case.
We propose an information-theoretic safe exploration criterion that directly exploits the GP posterior to identify the most informative safe parameters to evaluate.
arXiv Detail & Related papers (2024-02-23T14:31:10Z) - Likelihood Ratio Confidence Sets for Sequential Decision Making [51.66638486226482]
We revisit the likelihood-based inference principle and propose to use likelihood ratios to construct valid confidence sequences.
Our method is especially suitable for problems with well-specified likelihoods.
We show how to provably choose the best sequence of estimators and shed light on connections to online convex optimization.
arXiv Detail & Related papers (2023-11-08T00:10:21Z) - Equation Discovery with Bayesian Spike-and-Slab Priors and Efficient Kernels [57.46832672991433]
We propose a novel equation discovery method based on Kernel learning and BAyesian Spike-and-Slab priors (KBASS)
We use kernel regression to estimate the target function, which is flexible, expressive, and more robust to data sparsity and noises.
We develop an expectation-propagation expectation-maximization algorithm for efficient posterior inference and function estimation.
arXiv Detail & Related papers (2023-10-09T03:55:09Z) - On the Sublinear Regret of GP-UCB [58.25014663727544]
We show that the Gaussian Process Upper Confidence Bound (GP-UCB) algorithm enjoys nearly optimal regret rates.
Our improvements rely on a key technical contribution -- regularizing kernel ridge estimators in proportion to the smoothness of the underlying kernel.
arXiv Detail & Related papers (2023-07-14T13:56:11Z) - Guided Deep Kernel Learning [42.53025115287688]
We present a novel approach for learning deep kernels by utilizing infinite-width neural networks.
Our approach harnesses the reliable uncertainty estimation of the NNGPs to adapt the DKL target confidence when it encounters novel data points.
arXiv Detail & Related papers (2023-02-19T13:37:34Z) - Kernel Mean Estimation by Marginalized Corrupted Distributions [96.9272743070371]
Estimating the kernel mean in a kernel Hilbert space is a critical component in many kernel learning algorithms.
We present a new kernel mean estimator, called the marginalized kernel mean estimator, which estimates kernel mean under the corrupted distribution.
arXiv Detail & Related papers (2021-07-10T15:11:28Z) - The Promises and Pitfalls of Deep Kernel Learning [13.487684503022063]
We identify pathological behavior, including overfitting, on a simple toy example.
We explore this pathology, explaining its origins and considering how it applies to real datasets.
We find that a fully Bayesian treatment of deep kernel learning can rectify this overfitting and obtain the desired performance improvements.
arXiv Detail & Related papers (2021-02-24T07:56:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.