Neural Networks for Parameter Estimation in Intractable Models

• URL: http://arxiv.org/abs/2107.14346v1
• Date: Thu, 29 Jul 2021 21:59:48 GMT
• Title: Neural Networks for Parameter Estimation in Intractable Models
• Authors: Amanda Lenzi, Julie Bessac, Johann Rudi and Michael L. Stein
• Abstract summary: We show how to estimate parameters from max-stable processes, where inference is exceptionally challenging. We use data from model simulations as input and train deep neural networks to learn statistical parameters.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose to use deep learning to estimate parameters in statistical models when standard likelihood estimation methods are computationally infeasible. We show how to estimate parameters from max-stable processes, where inference is exceptionally challenging even with small datasets but simulation is straightforward. We use data from model simulations as input and train deep neural networks to learn statistical parameters. Our neural-network-based method provides a competitive alternative to current approaches, as demonstrated by considerable accuracy and computational time improvements. It serves as a proof of concept for deep learning in statistical parameter estimation and can be extended to other estimation problems.

Related papers

• Diffusion posterior sampling for simulation-based inference in tall data settings [53.17563688225137]
  Simulation-based inference ( SBI) is capable of approximating the posterior distribution that relates input parameters to a given observation. In this work, we consider a tall data extension in which multiple observations are available to better infer the parameters of the model. We compare our method to recently proposed competing approaches on various numerical experiments and demonstrate its superiority in terms of numerical stability and computational cost.
  arXiv  Detail & Related papers  (2024-04-11T09:23:36Z)
• Towards black-box parameter estimation [0.0]
  We develop new black-box procedures to estimate parameters of statistical models based on weak parameter structure assumptions. For well-structured likelihoods with frequent occurrences, this is achieved by pre-training a deep neural network on an extensive simulated database.
  arXiv  Detail & Related papers  (2023-03-27T09:39:38Z)
• New Machine Learning Techniques for Simulation-Based Inference: InferoStatic Nets, Kernel Score Estimation, and Kernel Likelihood Ratio Estimation [4.415977307120616]
  We propose a machine-learning approach to model the score and likelihood ratio estimators in cases when the probability density can be sampled but not computed directly. We introduce new strategies, respectively called Kernel Score Estimation (KSE) and Kernel Likelihood Ratio Estimation (KLRE) to learn the score and the likelihood ratio functions from simulated data.
  arXiv  Detail & Related papers  (2022-10-04T15:22:56Z)
• Learning to Learn with Generative Models of Neural Network Checkpoints [71.06722933442956]
  We construct a dataset of neural network checkpoints and train a generative model on the parameters. We find that our approach successfully generates parameters for a wide range of loss prompts. We apply our method to different neural network architectures and tasks in supervised and reinforcement learning.
  arXiv  Detail & Related papers  (2022-09-26T17:59:58Z)
• DeepBayes -- an estimator for parameter estimation in stochastic nonlinear dynamical models [11.917949887615567]
  We propose DeepBayes estimators that leverage the power of deep recurrent neural networks in learning an estimator. The deep recurrent neural network architectures can be trained offline and ensure significant time savings during inference. We demonstrate the applicability of our proposed method on different example models and perform detailed comparisons with state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-05-04T18:12:17Z)
• Learning Summary Statistics for Bayesian Inference with Autoencoders [58.720142291102135]
  We use the inner dimension of deep neural network based Autoencoders as summary statistics. To create an incentive for the encoder to encode all the parameter-related information but not the noise, we give the decoder access to explicit or implicit information that has been used to generate the training data.
  arXiv  Detail & Related papers  (2022-01-28T12:00:31Z)
• Physics-constrained deep neural network method for estimating parameters in a redox flow battery [68.8204255655161]
  We present a physics-constrained deep neural network (PCDNN) method for parameter estimation in the zero-dimensional (0D) model of the vanadium flow battery (VRFB) We show that the PCDNN method can estimate model parameters for a range of operating conditions and improve the 0D model prediction of voltage. We also demonstrate that the PCDNN approach has an improved generalization ability for estimating parameter values for operating conditions not used in the training.
  arXiv  Detail & Related papers  (2021-06-21T23:42:58Z)
• MINIMALIST: Mutual INformatIon Maximization for Amortized Likelihood Inference from Sampled Trajectories [61.3299263929289]
  Simulation-based inference enables learning the parameters of a model even when its likelihood cannot be computed in practice. One class of methods uses data simulated with different parameters to infer an amortized estimator for the likelihood-to-evidence ratio. We show that this approach can be formulated in terms of mutual information between model parameters and simulated data.
  arXiv  Detail & Related papers  (2021-06-03T12:59:16Z)
• Statistical model-based evaluation of neural networks [74.10854783437351]
  We develop an experimental setup for the evaluation of neural networks (NNs) The setup helps to benchmark a set of NNs vis-a-vis minimum-mean-square-error (MMSE) performance bounds. This allows us to test the effects of training data size, data dimension, data geometry, noise, and mismatch between training and testing conditions.
  arXiv  Detail & Related papers  (2020-11-18T00:33:24Z)
• Neural Approximate Sufficient Statistics for Implicit Models [34.44047460667847]
  We frame the task of constructing sufficient statistics as learning mutual information maximizing representations of the data with the help of deep neural networks. We apply our approach to both traditional approximate Bayesian computation and recent neural likelihood methods, boosting their performance on a range of tasks.
  arXiv  Detail & Related papers  (2020-10-20T07:11:40Z)
• BayesFlow: Learning complex stochastic models with invertible neural networks [3.1498833540989413]
  We propose a novel method for globally amortized Bayesian inference based on invertible neural networks. BayesFlow incorporates a summary network trained to embed the observed data into maximally informative summary statistics. We demonstrate the utility of BayesFlow on challenging intractable models from population dynamics, epidemiology, cognitive science and ecology.
  arXiv  Detail & Related papers  (2020-03-13T13:39:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.