Kernel-based guarantees for nonlinear parametric models in Bayesian optimization

• URL: http://arxiv.org/abs/2605.13160v1
• Date: Wed, 13 May 2026 08:24:55 GMT
• Title: Kernel-based guarantees for nonlinear parametric models in Bayesian optimization
• Abstract summary: We develop a kernel based framework for analyzing regularized nonlinear parametric models trained on adaptively collected data.<n>We show how these bounds can support convergence guarantees for nonlinear acquisition and surrogate models.
• Score: 5.191484562883949
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern Bayesian optimization and adaptive sampling methods increasingly rely on nonlinear parametric models, yet theoretical guarantees for such models under adaptive data collection remain limited. Existing analyses largely focus on Gaussian processes, kernel machines, linear models, or linearized neural approximations, leaving a gap between theory and the nonlinear models used in practice. We develop a kernel based framework for analyzing regularized nonlinear parametric models trained on adaptively collected data. Our approach uses kernels over the parameter space to induce reproducing kernel Hilbert space structures over the corresponding model class, yielding confidence bounds for models trained with broad classes of regularized convex losses. We show how these bounds can support convergence guarantees for nonlinear acquisition and surrogate models, including randomized regularized policies that select points by maximizing a trained random model. These results provide a unified route to analyzing nonlinear parametric models in Bayesian optimization and related adaptive optimization settings.

Related papers

• Nonparametric Data Attribution for Diffusion Models [57.820618036556084]
  Data attribution for generative models seeks to quantify the influence of individual training examples on model outputs.<n>We propose a nonparametric attribution method that operates entirely on data, measuring influence via patch-level similarity between generated and training images.
  arXiv  Detail & Related papers  (2025-10-16T03:37:16Z)
• A Framework for Nonstationary Gaussian Processes with Neural Network Parameters [0.8057006406834466]
  We propose a framework that uses nonstationary kernels whose parameters vary across the feature space, modeling these parameters as the output of a neural network.<n>Our method clearly describes the behavior of the nonstationary parameters and is compatible with approximation methods for scaling to large datasets.<n>We test a nonstationary variance and noise variant of our method on several machine learning datasets and find that it achieves better accuracy and log-score than both a stationary model and a hierarchical model approximated with variational inference.
  arXiv  Detail & Related papers  (2025-07-16T14:09:49Z)
• An Iterative Bayesian Approach for System Identification based on Linear Gaussian Models [70.75865595489691]
  We tackle the problem of system identification, where we select inputs, observe the corresponding outputs from the true system, and optimize the parameters of our model to best fit the data.<n>Our approach only requires input-output data from the system and first-order information of the model with respect to the parameters.
  arXiv  Detail & Related papers  (2025-01-28T01:57:51Z)
• Gaussian Process-Based Prediction and Control of Hammerstein-Wiener Systems [0.0]
  Data-driven prediction algorithms have been developed for structured nonlinear systems based on Willems' fundamental lemma.<n>Existing frameworks cannot treat output nonlinearities and require a dictionary of basis functions for Hammerstein systems.<n>In this work, an implicit predictor structure is considered, leveraging the multi-step-ahead ARX structure for the linear part of the model.
  arXiv  Detail & Related papers  (2025-01-27T08:18:14Z)
• Adaptive Conformal Inference by Betting [51.272991377903274]
  We consider the problem of adaptive conformal inference without any assumptions about the data generating process.<n>Existing approaches for adaptive conformal inference are based on optimizing the pinball loss using variants of online gradient descent.<n>We propose a different approach for adaptive conformal inference that leverages parameter-free online convex optimization techniques.
  arXiv  Detail & Related papers  (2024-12-26T18:42:08Z)
• Adaptive Nonparametric Perturbations of Parametric Bayesian Models [33.85958872117418]
  We study nonparametrically perturbed parametric (NPP) Bayesian models, in which a parametric Bayesian model is relaxed via a distortion of its likelihood.<n>We show that NPP models can offer the robustness of non models while retaining the data efficiency of parametric models.<n>We demonstrate our method by estimating causal effects of gene expression from single cell RNA sequencing data.
  arXiv  Detail & Related papers  (2024-12-14T05:06:38Z)
• Bayesian Inference for Consistent Predictions in Overparameterized Nonlinear Regression [0.0]
  This study explores the predictive properties of over parameterized nonlinear regression within the Bayesian framework. Posterior contraction is established for generalized linear and single-neuron models with Lipschitz continuous activation functions. The proposed method was validated via numerical simulations and a real data application.
  arXiv  Detail & Related papers  (2024-04-06T04:22:48Z)
• The Convex Landscape of Neural Networks: Characterizing Global Optima and Stationary Points via Lasso Models [75.33431791218302]
  Deep Neural Network Network (DNN) models are used for programming purposes. In this paper we examine the use of convex neural recovery models. We show that all the stationary non-dimensional objective objective can be characterized as the standard a global subsampled convex solvers program. We also show that all the stationary non-dimensional objective objective can be characterized as the standard a global subsampled convex solvers program.
  arXiv  Detail & Related papers  (2023-12-19T23:04:56Z)
• Hessian Eigenspectra of More Realistic Nonlinear Models [73.31363313577941]
  We make a emphprecise characterization of the Hessian eigenspectra for a broad family of nonlinear models. Our analysis takes a step forward to identify the origin of many striking features observed in more complex machine learning models.
  arXiv  Detail & Related papers  (2021-03-02T06:59:52Z)
• LQF: Linear Quadratic Fine-Tuning [114.3840147070712]
  We present the first method for linearizing a pre-trained model that achieves comparable performance to non-linear fine-tuning. LQF consists of simple modifications to the architecture, loss function and optimization typically used for classification.
  arXiv  Detail & Related papers  (2020-12-21T06:40:20Z)
• Uncertainty Modelling in Risk-averse Supply Chain Systems Using Multi-objective Pareto Optimization [0.0]
  One of the arduous tasks in supply chain modelling is to build robust models against irregular variations. We have introduced a novel methodology namely, Pareto Optimization to handle uncertainties and bound the entropy of such uncertainties by explicitly modelling them under some apriori assumptions.
  arXiv  Detail & Related papers  (2020-04-24T21:04:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.