Bayesian Semi-structured Subspace Inference

• URL: http://arxiv.org/abs/2401.12950v1
• Date: Tue, 23 Jan 2024 18:15:58 GMT
• Title: Bayesian Semi-structured Subspace Inference
• Authors: Daniel Dold, David R\"ugamer, Beate Sick, Oliver D\"urr
• Abstract summary: Semi-structured regression models enable the joint modeling of interpretable structured and complex unstructured feature effects. We present a Bayesian approximation for semi-structured regression models using subspace inference. Our approach exhibits competitive predictive performance across simulated and real-world datasets.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Semi-structured regression models enable the joint modeling of interpretable structured and complex unstructured feature effects. The structured model part is inspired by statistical models and can be used to infer the input-output relationship for features of particular importance. The complex unstructured part defines an arbitrary deep neural network and thereby provides enough flexibility to achieve competitive prediction performance. While these models can also account for aleatoric uncertainty, there is still a lack of work on accounting for epistemic uncertainty. In this paper, we address this problem by presenting a Bayesian approximation for semi-structured regression models using subspace inference. To this end, we extend subspace inference for joint posterior sampling from a full parameter space for structured effects and a subspace for unstructured effects. Apart from this hybrid sampling scheme, our method allows for tunable complexity of the subspace and can capture multiple minima in the loss landscape. Numerical experiments validate our approach's efficacy in recovering structured effect parameter posteriors in semi-structured models and approaching the full-space posterior distribution of MCMC for increasing subspace dimension. Further, our approach exhibits competitive predictive performance across simulated and real-world datasets.

Related papers

• A spatiotemporal deep learning framework for prediction of crack dynamics in heterogeneous solids: efficient mapping of concrete microstructures to its fracture properties [0.0]
  A deep learning framework is capable of 2D full-field prediction of fracture in concrete mesostructures. A convolutional neural network is developed which can predict the averaged stress-strain curve of mesostructures. The UNet modeling framework, which comprises an encoder-decoder section with skip connections, is used as the deep learning surrogate model.
  arXiv  Detail & Related papers  (2024-07-22T14:28:46Z)
• The Risk of Federated Learning to Skew Fine-Tuning Features and Underperform Out-of-Distribution Robustness [50.52507648690234]
  Federated learning has the risk of skewing fine-tuning features and compromising the robustness of the model. We introduce three robustness indicators and conduct experiments across diverse robust datasets. Our approach markedly enhances the robustness across diverse scenarios, encompassing various parameter-efficient fine-tuning methods.
  arXiv  Detail & Related papers  (2024-01-25T09:18:51Z)
• BART-SIMP: a novel framework for flexible spatial covariate modeling and prediction using Bayesian additive regression trees [0.0]
  We investigate a novel combination of a Gaussian process spatial model and a Bayesian Additive Regression Tree (BART) model. The computational burden of the approach is reduced by combining Markov chain Monte Carlo with the Integrated Nested Laplace Approximation (INLA) technique. We study the performance of the method via simulations and use the model to predict anthropometric responses, collected via household cluster samples in Kenya.
  arXiv  Detail & Related papers  (2023-09-23T05:35:17Z)
• Structured Radial Basis Function Network: Modelling Diversity for Multiple Hypotheses Prediction [51.82628081279621]
  Multi-modal regression is important in forecasting nonstationary processes or with a complex mixture of distributions. A Structured Radial Basis Function Network is presented as an ensemble of multiple hypotheses predictors for regression problems. It is proved that this structured model can efficiently interpolate this tessellation and approximate the multiple hypotheses target distribution.
  arXiv  Detail & Related papers  (2023-09-02T01:27:53Z)
• Hierarchical Neural Simulation-Based Inference Over Event Ensembles [0.4515457784397788]
  We introduce approaches for dataset-wide probabilistic inference in cases where the likelihood is intractable. We construct neural estimators for the likelihood(-ratio) or posterior and show that explicitly accounting for the model's hierarchical structure can lead to significantly tighter parameter constraints.
  arXiv  Detail & Related papers  (2023-06-21T21:50:42Z)
• Variational Inference for Bayesian Neural Networks under Model and Parameter Uncertainty [12.211659310564425]
  We apply the concept of model uncertainty as a framework for structural learning in BNNs. We suggest an adaptation of a scalable variational inference approach with reparametrization of marginal inclusion probabilities.
  arXiv  Detail & Related papers  (2023-05-01T16:38:17Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• MINIMALIST: Mutual INformatIon Maximization for Amortized Likelihood Inference from Sampled Trajectories [61.3299263929289]
  Simulation-based inference enables learning the parameters of a model even when its likelihood cannot be computed in practice. One class of methods uses data simulated with different parameters to infer an amortized estimator for the likelihood-to-evidence ratio. We show that this approach can be formulated in terms of mutual information between model parameters and simulated data.
  arXiv  Detail & Related papers  (2021-06-03T12:59:16Z)
• Supporting Optimal Phase Space Reconstructions Using Neural Network Architecture for Time Series Modeling [68.8204255655161]
  We propose an artificial neural network with a mechanism to implicitly learn the phase spaces properties. Our approach is either as competitive as or better than most state-of-the-art strategies.
  arXiv  Detail & Related papers  (2020-06-19T21:04:47Z)
• Generalising Recursive Neural Models by Tensor Decomposition [12.069862650316262]
  We introduce a general approach to model aggregation of structural context leveraging a tensor-based formulation. We show how the exponential growth in the size of the parameter space can be controlled through an approximation based on the Tucker decomposition. By this means, we can effectively regulate the trade-off between expressivity of the encoding, controlled by the hidden size, computational complexity and model generalisation.
  arXiv  Detail & Related papers  (2020-06-17T17:28:19Z)
• Semiparametric Bayesian Forecasting of Spatial Earthquake Occurrences [77.68028443709338]
  We propose a fully Bayesian formulation of the Epidemic Type Aftershock Sequence (ETAS) model. The occurrence of the mainshock earthquakes in a geographical region is assumed to follow an inhomogeneous spatial point process.
  arXiv  Detail & Related papers  (2020-02-05T10:11:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.