Consistency Models for Scalable and Fast Simulation-Based Inference
- URL: http://arxiv.org/abs/2312.05440v2
- Date: Tue, 27 Feb 2024 19:21:41 GMT
- Title: Consistency Models for Scalable and Fast Simulation-Based Inference
- Authors: Marvin Schmitt, Valentin Pratz, Ullrich K\"othe, Paul-Christian
B\"urkner, Stefan T Radev
- Abstract summary: We present CMPE, a new free-form conditional sampler for scalable, fast, and amortized neural networks.
It essentially distills a continuous probability flow and enables rapid few-shot inference with an unconstrained architecture.
It achieves competitive performance in a high-dimensional Bayesian denoising experiment and in estimating a computationally demanding multi-scale model of tumor spheroid growth.
- Score: 1.907072234794597
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Simulation-based inference (SBI) is constantly in search of more expressive
algorithms for accurately inferring the parameters of complex models from noisy
data. We present consistency models for neural posterior estimation (CMPE), a
new free-form conditional sampler for scalable, fast, and amortized SBI with
generative neural networks. CMPE combines the advantages of normalizing flows
and flow matching methods into a single generative architecture: It essentially
distills a continuous probability flow and enables rapid few-shot inference
with an unconstrained architecture that can be tailored to the structure of the
estimation problem. Our empirical evaluation demonstrates that CMPE not only
outperforms current state-of-the-art algorithms on three hard low-dimensional
problems but also achieves competitive performance in a high-dimensional
Bayesian denoising experiment and in estimating a computationally demanding
multi-scale model of tumor spheroid growth.
Related papers
- All-in-one simulation-based inference [19.41881319338419]
We present a new amortized inference method -- the Simformer -- which overcomes current limitations.
The Simformer outperforms current state-of-the-art amortized inference approaches on benchmark tasks.
It can be applied to models with function-valued parameters, it can handle inference scenarios with missing or unstructured data, and it can sample arbitrary conditionals of the joint distribution of parameters and data.
arXiv Detail & Related papers (2024-04-15T10:12:33Z) - Diffusion posterior sampling for simulation-based inference in tall data settings [53.17563688225137]
Simulation-based inference ( SBI) is capable of approximating the posterior distribution that relates input parameters to a given observation.
In this work, we consider a tall data extension in which multiple observations are available to better infer the parameters of the model.
We compare our method to recently proposed competing approaches on various numerical experiments and demonstrate its superiority in terms of numerical stability and computational cost.
arXiv Detail & Related papers (2024-04-11T09:23:36Z) - Online Variational Sequential Monte Carlo [49.97673761305336]
We build upon the variational sequential Monte Carlo (VSMC) method, which provides computationally efficient and accurate model parameter estimation and Bayesian latent-state inference.
Online VSMC is capable of performing efficiently, entirely on-the-fly, both parameter estimation and particle proposal adaptation.
arXiv Detail & Related papers (2023-12-19T21:45:38Z) - Flow Matching for Scalable Simulation-Based Inference [20.182658224439688]
Flow matching posterior estimation (FMPE) is a technique for simulation-based inference ( SBI) using continuous normalizing flows.
We show that FMPE achieves competitive performance on an established SBI benchmark, and then demonstrate its improved scalability on a challenging scientific problem.
arXiv Detail & Related papers (2023-05-26T18:00:01Z) - Multi-fidelity Hierarchical Neural Processes [79.0284780825048]
Multi-fidelity surrogate modeling reduces the computational cost by fusing different simulation outputs.
We propose Multi-fidelity Hierarchical Neural Processes (MF-HNP), a unified neural latent variable model for multi-fidelity surrogate modeling.
We evaluate MF-HNP on epidemiology and climate modeling tasks, achieving competitive performance in terms of accuracy and uncertainty estimation.
arXiv Detail & Related papers (2022-06-10T04:54:13Z) - Efficient hierarchical Bayesian inference for spatio-temporal regression
models in neuroimaging [6.512092052306553]
Examples include M/EEG inverse problems, encoding neural models for task-based fMRI analyses, and temperature monitoring schemes.
We devise a novel hierarchical flexible Bayesian framework within which the intrinsic-temporal dynamics of model parameters and noise are modeled.
arXiv Detail & Related papers (2021-11-02T15:50:01Z) - Scalable Spatiotemporally Varying Coefficient Modelling with Bayesian Kernelized Tensor Regression [17.158289775348063]
Kernelized tensor Regression (BKTR) can be considered a new and scalable approach to modeling processes with low-rank cotemporal structure.
We conduct extensive experiments on both synthetic and real-world data sets, and our results confirm the superior performance and efficiency of BKTR for model estimation and inference.
arXiv Detail & Related papers (2021-08-31T19:22:23Z) - Closed-form Continuous-Depth Models [99.40335716948101]
Continuous-depth neural models rely on advanced numerical differential equation solvers.
We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
arXiv Detail & Related papers (2021-06-25T22:08:51Z) - Sparse Flows: Pruning Continuous-depth Models [107.98191032466544]
We show that pruning improves generalization for neural ODEs in generative modeling.
We also show that pruning finds minimal and efficient neural ODE representations with up to 98% less parameters compared to the original network, without loss of accuracy.
arXiv Detail & Related papers (2021-06-24T01:40:17Z) - Deep Bayesian Active Learning for Accelerating Stochastic Simulation [74.58219903138301]
Interactive Neural Process (INP) is a deep active learning framework for simulations and with active learning approaches.
For active learning, we propose a novel acquisition function, Latent Information Gain (LIG), calculated in the latent space of NP based models.
The results demonstrate STNP outperforms the baselines in the learning setting and LIG achieves the state-of-the-art for active learning.
arXiv Detail & Related papers (2021-06-05T01:31:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.