Gaussian Process Models with Low-Rank Correlation Matrices for Both Continuous and Categorical Inputs

• URL: http://arxiv.org/abs/2010.02574v1
• Date: Tue, 6 Oct 2020 09:38:35 GMT
• Title: Gaussian Process Models with Low-Rank Correlation Matrices for Both Continuous and Categorical Inputs
• Authors: Dominik Kirchhoff, Sonja Kuhnt
• Abstract summary: We introduce a method that uses low-rank approximations of cross-correlation matrices in mixed continuous and categorical Gaussian Process models. Low-Rank Correlation (LRC) offers the ability to flexibly adapt the number of parameters to the problem at hand by choosing an appropriate rank of the approximation.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce a method that uses low-rank approximations of cross-correlation matrices in mixed continuous and categorical Gaussian Process models. This new method -- called Low-Rank Correlation (LRC) -- offers the ability to flexibly adapt the number of parameters to the problem at hand by choosing an appropriate rank of the approximation. Furthermore, we present a systematic approach of defining test functions that can be used for assessing the accuracy of models or optimization methods that are concerned with both continuous and categorical inputs. We compare LRC to existing approaches of modeling the cross-correlation matrix. It turns out that the new approach performs well in terms of estimation of cross-correlations and response surface prediction. Therefore, LRC is a flexible and useful addition to existing methods, especially for increasing numbers of combinations of levels of the categorical inputs.

Related papers

• Hyperparameter Estimation for Sparse Bayesian Learning Models [1.0172874946490507]
  Aparse Bayesian Learning (SBL) models are extensively used in signal processing and machine learning for promoting sparsity through hierarchical priors. This paper presents a framework for the improvement of SBL models for various objective functions. A novel algorithm is introduced showing enhanced efficiency, especially under signal noise ratios.
  arXiv  Detail & Related papers  (2024-01-04T21:24:01Z)
• Score-based Continuous-time Discrete Diffusion Models [102.65769839899315]
  We extend diffusion models to discrete variables by introducing a Markov jump process where the reverse process denoises via a continuous-time Markov chain. We show that an unbiased estimator can be obtained via simple matching the conditional marginal distributions. We demonstrate the effectiveness of the proposed method on a set of synthetic and real-world music and image benchmarks.
  arXiv  Detail & Related papers  (2022-11-30T05:33:29Z)
• Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition [54.07797071198249]
  We introduce a new scalable variational Gaussian process approximation which provides a high fidelity approximation while retaining general applicability. We demonstrate that, on a range of regression and classification problems, our approach can exploit input space symmetries such as translations and reflections. Notably, our approach achieves state-of-the-art results on CIFAR-10 among pure GP models.
  arXiv  Detail & Related papers  (2021-06-10T18:17:57Z)
• MINIMALIST: Mutual INformatIon Maximization for Amortized Likelihood Inference from Sampled Trajectories [61.3299263929289]
  Simulation-based inference enables learning the parameters of a model even when its likelihood cannot be computed in practice. One class of methods uses data simulated with different parameters to infer an amortized estimator for the likelihood-to-evidence ratio. We show that this approach can be formulated in terms of mutual information between model parameters and simulated data.
  arXiv  Detail & Related papers  (2021-06-03T12:59:16Z)
• Active and sparse methods in smoothed model checking [2.28438857884398]
  We consider extensions to smoothed model checking based on sparse variational methods and active learning. Online extensions of sparse variational Gaussian process inference algorithms are demonstrated to provide a scalable method for implementing active learning approaches for smoothed model checking.
  arXiv  Detail & Related papers  (2021-04-20T13:03:25Z)
• Piecewise linear regression and classification [0.20305676256390928]
  This paper proposes a method for solving multivariate regression and classification problems using piecewise linear predictors. A Python implementation of the algorithm described in this paper is available at http://cse.lab.imtlucca.it/bemporad/parc.
  arXiv  Detail & Related papers  (2021-03-10T17:07:57Z)
• Autoregressive Score Matching [113.4502004812927]
  We propose autoregressive conditional score models (AR-CSM) where we parameterize the joint distribution in terms of the derivatives of univariable log-conditionals (scores) For AR-CSM models, this divergence between data and model distributions can be computed and optimized efficiently, requiring no expensive sampling or adversarial training. We show with extensive experimental results that it can be applied to density estimation on synthetic data, image generation, image denoising, and training latent variable models with implicit encoders.
  arXiv  Detail & Related papers  (2020-10-24T07:01:24Z)
• Estimation of Switched Markov Polynomial NARX models [75.91002178647165]
  We identify a class of models for hybrid dynamical systems characterized by nonlinear autoregressive (NARX) components. The proposed approach is demonstrated on a SMNARX problem composed by three nonlinear sub-models with specific regressors.
  arXiv  Detail & Related papers  (2020-09-29T15:00:47Z)
• Additive interaction modelling using I-priors [0.571097144710995]
  We introduce a parsimonious specification of models with interactions, which has two benefits. It reduces the number of scale parameters and thus facilitates the estimation of models with interactions.
  arXiv  Detail & Related papers  (2020-07-30T22:52:22Z)
• Adaptive Correlated Monte Carlo for Contextual Categorical Sequence Generation [77.7420231319632]
  We adapt contextual generation of categorical sequences to a policy gradient estimator, which evaluates a set of correlated Monte Carlo (MC) rollouts for variance control. We also demonstrate the use of correlated MC rollouts for binary-tree softmax models, which reduce the high generation cost in large vocabulary scenarios.
  arXiv  Detail & Related papers  (2019-12-31T03:01:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.