Bayesian Optimization over Hybrid Spaces
- URL: http://arxiv.org/abs/2106.04682v1
- Date: Tue, 8 Jun 2021 20:47:21 GMT
- Title: Bayesian Optimization over Hybrid Spaces
- Authors: Aryan Deshwal, Syrine Belakaria, Janardhan Rao Doppa
- Abstract summary: We propose a novel approach to accurately model the complex interactions between discrete and continuous variables.
We develop a principled approach for constructing diffusion kernels over hybrid spaces by utilizing the additive kernel formulation.
Our experiments on synthetic and six diverse real-world benchmarks show that HyBO significantly outperforms the state-of-the-art methods.
- Score: 32.856318660282255
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We consider the problem of optimizing hybrid structures (mixture of discrete
and continuous input variables) via expensive black-box function evaluations.
This problem arises in many real-world applications. For example, in materials
design optimization via lab experiments, discrete and continuous variables
correspond to the presence/absence of primitive elements and their relative
concentrations respectively. The key challenge is to accurately model the
complex interactions between discrete and continuous variables. In this paper,
we propose a novel approach referred as Hybrid Bayesian Optimization (HyBO) by
utilizing diffusion kernels, which are naturally defined over continuous and
discrete variables. We develop a principled approach for constructing diffusion
kernels over hybrid spaces by utilizing the additive kernel formulation, which
allows additive interactions of all orders in a tractable manner. We
theoretically analyze the modeling strength of additive hybrid kernels and
prove that it has the universal approximation property. Our experiments on
synthetic and six diverse real-world benchmarks show that HyBO significantly
outperforms the state-of-the-art methods.
Related papers
- Approximation properties relative to continuous scale space for hybrid discretizations of Gaussian derivative operators [0.5439020425819]
This paper presents an analysis of properties of two hybrid discretization methods for Gaussian derivatives.
The motivation for studying these discretization methods is that in situations when multiple spatial derivatives of different order are needed at the same scale level, they can be computed significantly more efficiently.
arXiv Detail & Related papers (2024-05-08T14:44:34Z) - Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian
Mixture Models [59.331993845831946]
Diffusion models benefit from instillation of task-specific information into the score function to steer the sample generation towards desired properties.
This paper provides the first theoretical study towards understanding the influence of guidance on diffusion models in the context of Gaussian mixture models.
arXiv Detail & Related papers (2024-03-03T23:15:48Z) - Equivariant Flow Matching with Hybrid Probability Transport [69.11915545210393]
Diffusion Models (DMs) have demonstrated effectiveness in generating feature-rich geometries.
DMs typically suffer from unstable probability dynamics with inefficient sampling speed.
We introduce geometric flow matching, which enjoys the advantages of both equivariant modeling and stabilized probability dynamics.
arXiv Detail & Related papers (2023-12-12T11:13:13Z) - Gaussian Mixture Solvers for Diffusion Models [84.83349474361204]
We introduce a novel class of SDE-based solvers called GMS for diffusion models.
Our solver outperforms numerous SDE-based solvers in terms of sample quality in image generation and stroke-based synthesis.
arXiv Detail & Related papers (2023-11-02T02:05:38Z) - Generative Modeling on Manifolds Through Mixture of Riemannian Diffusion Processes [57.396578974401734]
We introduce a principled framework for building a generative diffusion process on general manifold.
Instead of following the denoising approach of previous diffusion models, we construct a diffusion process using a mixture of bridge processes.
We develop a geometric understanding of the mixture process, deriving the drift as a weighted mean of tangent directions to the data points.
arXiv Detail & Related papers (2023-10-11T06:04:40Z) - Non-separable Covariance Kernels for Spatiotemporal Gaussian Processes
based on a Hybrid Spectral Method and the Harmonic Oscillator [0.0]
We present a hybrid spectral approach for generating covariance kernels based on physical arguments.
We derive explicit relations for the covariance kernels in the three oscillator regimes (underdamping, critical damping, overdamping) and investigate their properties.
arXiv Detail & Related papers (2023-02-19T14:12:48Z) - Measuring dissimilarity with diffeomorphism invariance [94.02751799024684]
We introduce DID, a pairwise dissimilarity measure applicable to a wide range of data spaces.
We prove that DID enjoys properties which make it relevant for theoretical study and practical use.
arXiv Detail & Related papers (2022-02-11T13:51:30Z) - A comparison of mixed-variables Bayesian optimization approaches [0.0]
Most real optimization problems are defined over a mixed search space where the variables are both discrete and continuous.
In this article, costly mixed problems are approached through Gaussian processes where the discrete variables are relaxed into continuous latent variables.
In particular, the reformulation of the problem with continuous latent variables is put in competition with searches working directly in the mixed space.
arXiv Detail & Related papers (2021-10-30T09:26:34Z) - Heterogeneous Tensor Mixture Models in High Dimensions [5.656785831541303]
We consider the problem of jointly introducing a flexible high-dimensional tensor mixture model with heterogeneous covariances.
We show that our method converges geometrically to a neighborhood that is statistical of the true parameter.
Our analysis identifies important brain regions for diagnosis in an autism spectrum disorder.
arXiv Detail & Related papers (2021-04-15T21:06:16Z) - Mixed Variable Bayesian Optimization with Frequency Modulated Kernels [96.78099706164747]
We propose the frequency modulated (FM) kernel flexibly modeling dependencies among different types of variables.
BO-FM outperforms competitors including Regularized evolution(RE) and BOHB.
arXiv Detail & Related papers (2021-02-25T11:28:46Z) - Stochastic modeling of non-linear adsorption with Gaussian kernel
density estimators [2.7126936616742845]
Adsorption is a relevant process in many fields, such as product manufacturing or pollution remediation.
We have proposed a chemical diffusion-reaction model for the simulation ofAdsorption.
The proposed model can effectively reproduce the nonlinear behavior characteristic of the Langmuir and Freundlich isotherms.
arXiv Detail & Related papers (2020-04-14T12:21:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.