Bayesian Experimental Design for Implicit Models by Mutual Information Neural Estimation

• URL: http://arxiv.org/abs/2002.08129v3
• Date: Fri, 14 Aug 2020 15:04:46 GMT
• Title: Bayesian Experimental Design for Implicit Models by Mutual Information Neural Estimation
• Authors: Steven Kleinegesse and Michael U. Gutmann
• Abstract summary: Implicit models, where the data-generation distribution is intractable but sampling is possible, are ubiquitous in the natural sciences. A fundamental question is how to design experiments so that the collected data are most useful. For implicit models, however, this approach is severely hampered by the high computational cost of computing posteriors. We show that training a neural network to maximise a lower bound on MI allows us to jointly determine the optimal design and the posterior.
• Score: 16.844481439960663
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Implicit stochastic models, where the data-generation distribution is intractable but sampling is possible, are ubiquitous in the natural sciences. The models typically have free parameters that need to be inferred from data collected in scientific experiments. A fundamental question is how to design the experiments so that the collected data are most useful. The field of Bayesian experimental design advocates that, ideally, we should choose designs that maximise the mutual information (MI) between the data and the parameters. For implicit models, however, this approach is severely hampered by the high computational cost of computing posteriors and maximising MI, in particular when we have more than a handful of design variables to optimise. In this paper, we propose a new approach to Bayesian experimental design for implicit models that leverages recent advances in neural MI estimation to deal with these issues. We show that training a neural network to maximise a lower bound on MI allows us to jointly determine the optimal design and the posterior. Simulation studies illustrate that this gracefully extends Bayesian experimental design for implicit models to higher design dimensions.

Related papers

• Diffusion posterior sampling for simulation-based inference in tall data settings [53.17563688225137]
  Simulation-based inference ( SBI) is capable of approximating the posterior distribution that relates input parameters to a given observation. In this work, we consider a tall data extension in which multiple observations are available to better infer the parameters of the model. We compare our method to recently proposed competing approaches on various numerical experiments and demonstrate its superiority in terms of numerical stability and computational cost.
  arXiv  Detail & Related papers  (2024-04-11T09:23:36Z)
• Diffusion Model for Data-Driven Black-Box Optimization [54.25693582870226]
  We focus on diffusion models, a powerful generative AI technology, and investigate their potential for black-box optimization. We study two practical types of labels: 1) noisy measurements of a real-valued reward function and 2) human preference based on pairwise comparisons. Our proposed method reformulates the design optimization problem into a conditional sampling problem, which allows us to leverage the power of diffusion models.
  arXiv  Detail & Related papers  (2024-03-20T00:41:12Z)
• Online simulator-based experimental design for cognitive model selection [74.76661199843284]
  We propose BOSMOS: an approach to experimental design that can select between computational models without tractable likelihoods. In simulated experiments, we demonstrate that the proposed BOSMOS technique can accurately select models in up to 2 orders of magnitude less time than existing LFI alternatives.
  arXiv  Detail & Related papers  (2023-03-03T21:41:01Z)
• Design Amortization for Bayesian Optimal Experimental Design [70.13948372218849]
  We build off of successful variational approaches, which optimize a parameterized variational model with respect to bounds on the expected information gain (EIG) We present a novel neural architecture that allows experimenters to optimize a single variational model that can estimate the EIG for potentially infinitely many designs.
  arXiv  Detail & Related papers  (2022-10-07T02:12:34Z)
• Mixed Effects Neural ODE: A Variational Approximation for Analyzing the Dynamics of Panel Data [50.23363975709122]
  We propose a probabilistic model called ME-NODE to incorporate (fixed + random) mixed effects for analyzing panel data. We show that our model can be derived using smooth approximations of SDEs provided by the Wong-Zakai theorem. We then derive Evidence Based Lower Bounds for ME-NODE, and develop (efficient) training algorithms.
  arXiv  Detail & Related papers  (2022-02-18T22:41:51Z)
• Inverting brain grey matter models with likelihood-free inference: a tool for trustable cytoarchitecture measurements [62.997667081978825]
  characterisation of the brain grey matter cytoarchitecture with quantitative sensitivity to soma density and volume remains an unsolved challenge in dMRI. We propose a new forward model, specifically a new system of equations, requiring a few relatively sparse b-shells. We then apply modern tools from Bayesian analysis known as likelihood-free inference (LFI) to invert our proposed model.
  arXiv  Detail & Related papers  (2021-11-15T09:08:27Z)
• Gradient-based Bayesian Experimental Design for Implicit Models using Mutual Information Lower Bounds [20.393359858407162]
  We introduce a framework for Bayesian experimental design (BED) with implicit models, where the data-generating distribution is intractable but sampling from it is still possible. In order to find optimal experimental designs for such models, our approach maximises mutual information lower bounds that are parametrised by neural networks. By training a neural network on sampled data, we simultaneously update network parameters and designs using gradient-ascent.
  arXiv  Detail & Related papers  (2021-05-10T13:59:25Z)
• A Hybrid Gradient Method to Designing Bayesian Experiments for Implicit Models [3.437223569602425]
  The optimal design is usually achieved by maximizing the mutual information (MI) between the data and the model parameters. When the analytical expression of the MI is unavailable, e.g., having implicit models with intractable data distributions, a neural network-based lower bound of the MI was recently proposed and a gradient ascent method was used to maximize the lower bound. We propose a hybrid approach that leverages recent advances in variational MI estimator and evolution strategies (ES) combined with black-box gradient ascent (SGA) to maximize the MI lower bound.
  arXiv  Detail & Related papers  (2021-03-14T21:10:03Z)
• Sequential Bayesian Experimental Design for Implicit Models via Mutual Information [12.68659360172393]
  A class of models of particular interest for the natural and medical sciences are implicit models. We devise a novel sequential design framework for parameter estimation that uses the Mutual Information (MI) between model parameters and simulated data as a utility function. We find that our framework is efficient for the various implicit models tested, yielding accurate parameter estimates after only a few iterations.
  arXiv  Detail & Related papers  (2020-03-20T16:52:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.