Bayesian Optimization over Hybrid Spaces

• URL: http://arxiv.org/abs/2106.04682v1
• Date: Tue, 8 Jun 2021 20:47:21 GMT
• Title: Bayesian Optimization over Hybrid Spaces
• Authors: Aryan Deshwal, Syrine Belakaria, Janardhan Rao Doppa
• Abstract summary: We propose a novel approach to accurately model the complex interactions between discrete and continuous variables. We develop a principled approach for constructing diffusion kernels over hybrid spaces by utilizing the additive kernel formulation. Our experiments on synthetic and six diverse real-world benchmarks show that HyBO significantly outperforms the state-of-the-art methods.
• Score: 32.856318660282255
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We consider the problem of optimizing hybrid structures (mixture of discrete and continuous input variables) via expensive black-box function evaluations. This problem arises in many real-world applications. For example, in materials design optimization via lab experiments, discrete and continuous variables correspond to the presence/absence of primitive elements and their relative concentrations respectively. The key challenge is to accurately model the complex interactions between discrete and continuous variables. In this paper, we propose a novel approach referred as Hybrid Bayesian Optimization (HyBO) by utilizing diffusion kernels, which are naturally defined over continuous and discrete variables. We develop a principled approach for constructing diffusion kernels over hybrid spaces by utilizing the additive kernel formulation, which allows additive interactions of all orders in a tractable manner. We theoretically analyze the modeling strength of additive hybrid kernels and prove that it has the universal approximation property. Our experiments on synthetic and six diverse real-world benchmarks show that HyBO significantly outperforms the state-of-the-art methods.

Related papers

• Rethinking Diffusion Models with Symmetries through Canonicalization with Applications to Molecular Graph Generation [56.361076943802594]
  CanonFlow achieves state-of-the-art performance on the challenging GEOM-DRUG dataset, and the advantage remains large in few-step generation.
  arXiv  Detail & Related papers  (2026-02-16T18:58:55Z)
• Regime-Adaptive Bayesian Optimization via Dirichlet Process Mixtures of Gaussian Processes [14.367563771141592]
  RAMBO is a Dirichlet Process Mixture of Gaussian Processes that automatically discovers latent regimes during optimization.<n>We derive collapsed Gibbs sampling that analytically marginalizes latent functions for efficient inference.<n>Our acquisition functions decompose uncertainty into intra-regime and inter-regime components.
  arXiv  Detail & Related papers  (2026-01-27T20:45:50Z)
• Varying-Coefficient Mixture of Experts Model [0.0]
  We propose a Varying-Coefficient Mixture of Experts (VCMoE) model that allows all coefficient effects in both the gating functions and expert models to vary along an indexing variable.<n>We illustrate the proposed VCMoE model using a dataset of single gene expression in embryonic mice to characterize the temporal dynamics of the associations between the expression levels of genes Satb2 and Bcl11b.
  arXiv  Detail & Related papers  (2026-01-05T00:23:30Z)
• Foundations of Diffusion Models in General State Spaces: A Self-Contained Introduction [54.95522167029998]
  This article is a self-contained primer on diffusion over general state spaces.<n>We develop the discrete-time view (forward noising via Markov kernels and learned reverse dynamics) alongside its continuous-time limits.<n>A common variational treatment yields the ELBO that underpins standard training losses.
  arXiv  Detail & Related papers  (2025-12-04T18:55:36Z)
• MissHDD: Hybrid Deterministic Diffusion for Hetrogeneous Incomplete Data Imputation [4.935498694293104]
  We propose a hybrid deterministic diffusion framework that separates heterogeneous features into two complementary generative channels.<n>A continuous DDIM-based channel provides efficient and stable deterministic denoising for numerical variables.<n>A discrete latent-path diffusion channel, inspired by loopholing-based discrete diffusion, models categorical and discrete features without leaving their valid sample.<n>The two channels are trained under a unified conditional imputation objective, enabling coherent reconstruction of mixed-type incomplete data.
  arXiv  Detail & Related papers  (2025-11-18T14:44:49Z)
• Structure Language Models for Protein Conformation Generation [66.42864253026053]
  Traditional physics-based simulation methods often struggle with sampling equilibrium conformations. Deep generative models have shown promise in generating protein conformations as a more efficient alternative. We introduce Structure Language Modeling as a novel framework for efficient protein conformation generation.
  arXiv  Detail & Related papers  (2024-10-24T03:38:51Z)
• Approximation properties relative to continuous scale space for hybrid discretizations of Gaussian derivative operators [0.5439020425819]
  This paper presents an analysis of properties of two hybrid discretization methods for Gaussian derivatives. The motivation for studying these discretization methods is that in situations when multiple spatial derivatives of different order are needed at the same scale level, they can be computed significantly more efficiently.
  arXiv  Detail & Related papers  (2024-05-08T14:44:34Z)
• Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian Mixture Models [59.331993845831946]
  Diffusion models benefit from instillation of task-specific information into the score function to steer the sample generation towards desired properties. This paper provides the first theoretical study towards understanding the influence of guidance on diffusion models in the context of Gaussian mixture models.
  arXiv  Detail & Related papers  (2024-03-03T23:15:48Z)
• Gaussian Mixture Solvers for Diffusion Models [84.83349474361204]
  We introduce a novel class of SDE-based solvers called GMS for diffusion models. Our solver outperforms numerous SDE-based solvers in terms of sample quality in image generation and stroke-based synthesis.
  arXiv  Detail & Related papers  (2023-11-02T02:05:38Z)
• Generative Modeling on Manifolds Through Mixture of Riemannian Diffusion Processes [57.396578974401734]
  We introduce a principled framework for building a generative diffusion process on general manifold. Instead of following the denoising approach of previous diffusion models, we construct a diffusion process using a mixture of bridge processes. We develop a geometric understanding of the mixture process, deriving the drift as a weighted mean of tangent directions to the data points.
  arXiv  Detail & Related papers  (2023-10-11T06:04:40Z)
• Non-separable Covariance Kernels for Spatiotemporal Gaussian Processes based on a Hybrid Spectral Method and the Harmonic Oscillator [0.0]
  We present a hybrid spectral approach for generating covariance kernels based on physical arguments. We derive explicit relations for the covariance kernels in the three oscillator regimes (underdamping, critical damping, overdamping) and investigate their properties.
  arXiv  Detail & Related papers  (2023-02-19T14:12:48Z)
• Measuring dissimilarity with diffeomorphism invariance [94.02751799024684]
  We introduce DID, a pairwise dissimilarity measure applicable to a wide range of data spaces. We prove that DID enjoys properties which make it relevant for theoretical study and practical use.
  arXiv  Detail & Related papers  (2022-02-11T13:51:30Z)
• A comparison of mixed-variables Bayesian optimization approaches [0.0]
  Most real optimization problems are defined over a mixed search space where the variables are both discrete and continuous. In this article, costly mixed problems are approached through Gaussian processes where the discrete variables are relaxed into continuous latent variables. In particular, the reformulation of the problem with continuous latent variables is put in competition with searches working directly in the mixed space.
  arXiv  Detail & Related papers  (2021-10-30T09:26:34Z)
• Mixed Variable Bayesian Optimization with Frequency Modulated Kernels [96.78099706164747]
  We propose the frequency modulated (FM) kernel flexibly modeling dependencies among different types of variables. BO-FM outperforms competitors including Regularized evolution(RE) and BOHB.
  arXiv  Detail & Related papers  (2021-02-25T11:28:46Z)
• Learning Inconsistent Preferences with Gaussian Processes [14.64963271587818]
  We revisit widely used preferential Gaussian processes by Chu et al.(2005) and challenge their modelling assumption that imposes rankability of data items via latent utility function values. We propose a generalisation of pgp which can capture more expressive latent preferential structures in the data. Our experimental findings support the conjecture that violations of rankability are ubiquitous in real-world preferential data.
  arXiv  Detail & Related papers  (2020-06-06T11:57:45Z)
• Stochastic modeling of non-linear adsorption with Gaussian kernel density estimators [2.7126936616742845]
  Adsorption is a relevant process in many fields, such as product manufacturing or pollution remediation. We have proposed a chemical diffusion-reaction model for the simulation ofAdsorption. The proposed model can effectively reproduce the nonlinear behavior characteristic of the Langmuir and Freundlich isotherms.
  arXiv  Detail & Related papers  (2020-04-14T12:21:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.