On Estimating the Gradient of the Expected Information Gain in Bayesian Experimental Design

• URL: http://arxiv.org/abs/2308.09888v2
• Date: Tue, 12 Dec 2023 21:21:08 GMT
• Title: On Estimating the Gradient of the Expected Information Gain in Bayesian Experimental Design
• Authors: Ziqiao Ao, Jinglai Li
• Abstract summary: We develop methods for estimating the gradient of EIG, which combined with gradient descent algorithms, result in efficient optimization of EIG. Based on this, we propose two methods for estimating the EIG gradient, UEEG-MCMC that leverages posterior samples to estimate the EIG gradient, and BEEG-AP that focuses on achieving high simulation efficiency by repeatedly using parameter samples.
• Score: 5.874142059884521
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Bayesian Experimental Design (BED), which aims to find the optimal experimental conditions for Bayesian inference, is usually posed as to optimize the expected information gain (EIG). The gradient information is often needed for efficient EIG optimization, and as a result the ability to estimate the gradient of EIG is essential for BED problems. The primary goal of this work is to develop methods for estimating the gradient of EIG, which, combined with the stochastic gradient descent algorithms, result in efficient optimization of EIG. Specifically, we first introduce a posterior expected representation of the EIG gradient with respect to the design variables. Based on this, we propose two methods for estimating the EIG gradient, UEEG-MCMC that leverages posterior samples generated through Markov Chain Monte Carlo (MCMC) to estimate the EIG gradient, and BEEG-AP that focuses on achieving high simulation efficiency by repeatedly using parameter samples. Theoretical analysis and numerical studies illustrate that UEEG-MCMC is robust agains the actual EIG value, while BEEG-AP is more efficient when the EIG value to be optimized is small. Moreover, both methods show superior performance compared to several popular benchmarks in our numerical experiments.

Related papers

• Bayesian Experimental Design via Contrastive Diffusions [2.2186678387006435]
  Experimental Design (BOED) is a powerful tool to reduce the cost of running a sequence of experiments. We introduce an it expected posterior distribution with cost-effective properties and provide a tractable access to the EIG contrast. By incorporating generative models into the BOED framework, we expand its scope and its use in scenarios that were previously impractical.
  arXiv  Detail & Related papers  (2024-10-15T17:53:07Z)
• A Likelihood-Free Approach to Goal-Oriented Bayesian Optimal Experimental Design [0.0]
  We introduce LF-GO-OED (likelihood-free goal-oriented optimal experimental design), a computational method for conducting GO-OED with nonlinear observation and prediction models. It is specifically designed to accommodate implicit models, where the likelihood is intractable. The method is validated on benchmark problems with existing methods, and demonstrated on scientific applications of epidemiology and neural science.
  arXiv  Detail & Related papers  (2024-08-18T19:45:49Z)
• Design Amortization for Bayesian Optimal Experimental Design [70.13948372218849]
  We build off of successful variational approaches, which optimize a parameterized variational model with respect to bounds on the expected information gain (EIG) We present a novel neural architecture that allows experimenters to optimize a single variational model that can estimate the EIG for potentially infinitely many designs.
  arXiv  Detail & Related papers  (2022-10-07T02:12:34Z)
• Robust Expected Information Gain for Optimal Bayesian Experimental Design Using Ambiguity Sets [0.0]
  We define and analyze emphrobust expected information gain (REIG) REIG is a modification of the objective in EIG by minimizing an affine relaxation of EIG over an ambiguity set of perturbed distributions. We show that, when combined with a sampling-based approach to estimating EIG, REIG corresponds to a log-sum-exp' stabilization of the samples used to estimate EIG.
  arXiv  Detail & Related papers  (2022-05-20T01:07:41Z)
• AD-NEGF: An End-to-End Differentiable Quantum Transport Simulator for Sensitivity Analysis and Inverse Problems [14.955199623904157]
  We propose AD-NEGF, to our best knowledge the first end-to-end differentiable NEGF model for quantum transport simulations. We implement the entire numerical process in PyTorch, and design customized backward pass with implicit layer techniques. The proposed model is validated with applications in calculating differential physical quantities, empirical parameter fitting, and doping optimization.
  arXiv  Detail & Related papers  (2022-02-10T15:35:48Z)
• Cauchy-Schwarz Regularized Autoencoder [68.80569889599434]
  Variational autoencoders (VAE) are a powerful and widely-used class of generative models. We introduce a new constrained objective based on the Cauchy-Schwarz divergence, which can be computed analytically for GMMs. Our objective improves upon variational auto-encoding models in density estimation, unsupervised clustering, semi-supervised learning, and face analysis.
  arXiv  Detail & Related papers  (2021-01-06T17:36:26Z)
• Zeroth-Order Hybrid Gradient Descent: Towards A Principled Black-Box Optimization Framework [100.36569795440889]
  This work is on the iteration of zero-th-order (ZO) optimization which does not require first-order information. We show that with a graceful design in coordinate importance sampling, the proposed ZO optimization method is efficient both in terms of complexity as well as as function query cost.
  arXiv  Detail & Related papers  (2020-12-21T17:29:58Z)
• Bilevel Optimization: Convergence Analysis and Enhanced Design [63.64636047748605]
  Bilevel optimization is a tool for many machine learning problems. We propose a novel stoc-efficientgradient estimator named stoc-BiO.
  arXiv  Detail & Related papers  (2020-10-15T18:09:48Z)
• On the Convergence Rate of Projected Gradient Descent for a Back-Projection based Objective [58.33065918353532]
  We consider a back-projection based fidelity term as an alternative to the common least squares (LS) We show that using the BP term, rather than the LS term, requires fewer iterations of optimization algorithms.
  arXiv  Detail & Related papers  (2020-05-03T00:58:23Z)
• Improving Sampling Accuracy of Stochastic Gradient MCMC Methods via Non-uniform Subsampling of Gradients [54.90670513852325]
  We propose a non-uniform subsampling scheme to improve the sampling accuracy. EWSG is designed so that a non-uniform gradient-MCMC method mimics the statistical behavior of a batch-gradient-MCMC method. In our practical implementation of EWSG, the non-uniform subsampling is performed efficiently via a Metropolis-Hastings chain on the data index.
  arXiv  Detail & Related papers  (2020-02-20T18:56:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.