Scalable Semi-Modular Inference with Variational Meta-Posteriors

• URL: http://arxiv.org/abs/2204.00296v1
• Date: Fri, 1 Apr 2022 09:01:20 GMT
• Title: Scalable Semi-Modular Inference with Variational Meta-Posteriors
• Authors: Chris U. Carmona, Geoff K. Nicholls
• Abstract summary: The Cut posterior and Semi-Modular Inference are Generalised Bayes methods for Modular Bayesian evidence combination. We show that analysis of models with multiple cuts is feasible using a new Variational Meta-Posterior. This approximates a family of SMI posteriors indexed by $eta$ using a single set of variational parameters.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Cut posterior and related Semi-Modular Inference are Generalised Bayes methods for Modular Bayesian evidence combination. Analysis is broken up over modular sub-models of the joint posterior distribution. Model-misspecification in multi-modular models can be hard to fix by model elaboration alone and the Cut posterior and SMI offer a way round this. Information entering the analysis from misspecified modules is controlled by an influence parameter $\eta$ related to the learning rate. This paper contains two substantial new methods. First, we give variational methods for approximating the Cut and SMI posteriors which are adapted to the inferential goals of evidence combination. We parameterise a family of variational posteriors using a Normalising Flow for accurate approximation and end-to-end training. Secondly, we show that analysis of models with multiple cuts is feasible using a new Variational Meta-Posterior. This approximates a family of SMI posteriors indexed by $\eta$ using a single set of variational parameters.

Related papers

• SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models [85.67096251281191]
  We present an innovative approach to model fusion called zero-shot Sparse MIxture of Low-rank Experts (SMILE) construction. SMILE allows for the upscaling of source models into an MoE model without extra data or further training. We conduct extensive experiments across diverse scenarios, such as image classification and text generation tasks, using full fine-tuning and LoRA fine-tuning.
  arXiv  Detail & Related papers  (2024-08-19T17:32:15Z)
• EMR-Merging: Tuning-Free High-Performance Model Merging [55.03509900949149]
  We show that Elect, Mask & Rescale-Merging (EMR-Merging) shows outstanding performance compared to existing merging methods. EMR-Merging is tuning-free, thus requiring no data availability or any additional training while showing impressive performance.
  arXiv  Detail & Related papers  (2024-05-23T05:25:45Z)
• Bayesian learning of Causal Structure and Mechanisms with GFlowNets and Variational Bayes [51.84122462615402]
  We introduce a novel method to learn the structure and mechanisms of the causal model using Variational Bayes-DAG-GFlowNet. We extend the method of Bayesian causal structure learning using GFlowNets to learn the parameters of a linear-Gaussian model.
  arXiv  Detail & Related papers  (2022-11-04T21:57:39Z)
• Dynamically-Scaled Deep Canonical Correlation Analysis [77.34726150561087]
  Canonical Correlation Analysis (CCA) is a method for feature extraction of two views by finding maximally correlated linear projections of them. We introduce a novel dynamic scaling method for training an input-dependent canonical correlation model.
  arXiv  Detail & Related papers  (2022-03-23T12:52:49Z)
• Inverting brain grey matter models with likelihood-free inference: a tool for trustable cytoarchitecture measurements [62.997667081978825]
  characterisation of the brain grey matter cytoarchitecture with quantitative sensitivity to soma density and volume remains an unsolved challenge in dMRI. We propose a new forward model, specifically a new system of equations, requiring a few relatively sparse b-shells. We then apply modern tools from Bayesian analysis known as likelihood-free inference (LFI) to invert our proposed model.
  arXiv  Detail & Related papers  (2021-11-15T09:08:27Z)
• Transformation Models for Flexible Posteriors in Variational Bayes [0.0]
  In neural networks, variational inference is widely used to approximate difficult-to-compute posteriors. Transformation models are flexible enough to fit any distribution. TM-VI allows to accurately approximate complex posteriors in models with one parameter.
  arXiv  Detail & Related papers  (2021-06-01T14:43:47Z)
• Mutual Information Constraints for Monte-Carlo Objectives [21.70557526001205]
  A common failure mode of density models trained as variational autoencoders is to model the data without relying on their latent variables, rendering these variables useless. We weave these two strands of research together, specifically the tighter bounds of Monte-Carlo objectives and constraints on the mutual information between the observable and the latent variables. We construct estimators of the Kullback-Leibler divergence of the true posterior from the prior by recycling samples used in the objective, with which we train models of continuous and discrete latents at much improved rate-distortion and no posterior collapse.
  arXiv  Detail & Related papers  (2020-12-01T18:14:08Z)
• Meta-Learning with Shared Amortized Variational Inference [33.54630534228469]
  We propose a novel amortized variational inference scheme for an empirical Bayes meta-learning model. We learn the prior distribution over model parameters conditioned on limited training data using a variational autoencoder approach.
  arXiv  Detail & Related papers  (2020-08-27T10:28:13Z)
• Semi-Modular Inference: enhanced learning in multi-modular models by tempering the influence of components [0.0]
  We show existing Modular/Cut-model inference is coherent, and write down a new family of Semi-Modular Inference schemes. We give a meta-learning criterion and estimation procedure to choose the inference scheme. We illustrate our methods on two standard test cases from the literature and a motivating archaeological data set.
  arXiv  Detail & Related papers  (2020-03-15T11:55:55Z)
• Decision-Making with Auto-Encoding Variational Bayes [71.44735417472043]
  We show that a posterior approximation distinct from the variational distribution should be used for making decisions. Motivated by these theoretical results, we propose learning several approximate proposals for the best model. In addition to toy examples, we present a full-fledged case study of single-cell RNA sequencing.
  arXiv  Detail & Related papers  (2020-02-17T19:23:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.