Active Sequential Posterior Estimation for Sample-Efficient Simulation-Based Inference

• URL: http://arxiv.org/abs/2412.05590v1
• Date: Sat, 07 Dec 2024 08:57:26 GMT
• Title: Active Sequential Posterior Estimation for Sample-Efficient Simulation-Based Inference
• Authors: Sam Griesemer, Defu Cao, Zijun Cui, Carolina Osorio, Yan Liu,
• Abstract summary: We introduce sequential neural posterior estimation (ASNPE) ASNPE brings an active learning scheme into the inference loop to estimate the utility of simulation parameter candidates to the underlying probabilistic model. Our method outperforms well-tuned benchmarks and state-of-the-art posterior estimation methods on a large-scale real-world traffic network.
• Score: 12.019504660711231
• License:
• Abstract: Computer simulations have long presented the exciting possibility of scientific insight into complex real-world processes. Despite the power of modern computing, however, it remains challenging to systematically perform inference under simulation models. This has led to the rise of simulation-based inference (SBI), a class of machine learning-enabled techniques for approaching inverse problems with stochastic simulators. Many such methods, however, require large numbers of simulation samples and face difficulty scaling to high-dimensional settings, often making inference prohibitive under resource-intensive simulators. To mitigate these drawbacks, we introduce active sequential neural posterior estimation (ASNPE). ASNPE brings an active learning scheme into the inference loop to estimate the utility of simulation parameter candidates to the underlying probabilistic model. The proposed acquisition scheme is easily integrated into existing posterior estimation pipelines, allowing for improved sample efficiency with low computational overhead. We further demonstrate the effectiveness of the proposed method in the travel demand calibration setting, a high-dimensional inverse problem commonly requiring computationally expensive traffic simulators. Our method outperforms well-tuned benchmarks and state-of-the-art posterior estimation methods on a large-scale real-world traffic network, as well as demonstrates a performance advantage over non-active counterparts on a suite of SBI benchmark environments.

Related papers

• DSMoE: Matrix-Partitioned Experts with Dynamic Routing for Computation-Efficient Dense LLMs [70.91804882618243]
  This paper proposes DSMoE, a novel approach that achieves sparsification by partitioning pre-trained FFN layers into computational blocks. We implement adaptive expert routing using sigmoid activation and straight-through estimators, enabling tokens to flexibly access different aspects of model knowledge. Experiments on LLaMA models demonstrate that under equivalent computational constraints, DSMoE achieves superior performance compared to existing pruning and MoE approaches.
  arXiv  Detail & Related papers  (2025-02-18T02:37:26Z)
• Parallel simulation for sampling under isoperimetry and score-based diffusion models [56.39904484784127]
  As data size grows, reducing the iteration cost becomes an important goal. Inspired by the success of the parallel simulation of the initial value problem in scientific computation, we propose parallel Picard methods for sampling tasks. Our work highlights the potential advantages of simulation methods in scientific computation for dynamics-based sampling and diffusion models.
  arXiv  Detail & Related papers  (2024-12-10T11:50:46Z)
• Embed and Emulate: Contrastive representations for simulation-based inference [11.543221890134399]
  This paper introduces Embed and Emulate (E&E), a new simulation-based inference ( SBI) method based on contrastive learning. E&E learns a low-dimensional latent embedding of the data and a corresponding fast emulator in the latent space. We demonstrate superior performance over existing methods in a realistic, non-identifiable parameter estimation task.
  arXiv  Detail & Related papers  (2024-09-27T02:37:01Z)
• Diffusion posterior sampling for simulation-based inference in tall data settings [53.17563688225137]
  Simulation-based inference ( SBI) is capable of approximating the posterior distribution that relates input parameters to a given observation. In this work, we consider a tall data extension in which multiple observations are available to better infer the parameters of the model. We compare our method to recently proposed competing approaches on various numerical experiments and demonstrate its superiority in terms of numerical stability and computational cost.
  arXiv  Detail & Related papers  (2024-04-11T09:23:36Z)
• Surrogate Neural Networks for Efficient Simulation-based Trajectory Planning Optimization [28.292234483886947]
  This paper presents a novel methodology that uses surrogate models in the form of neural networks to reduce the computation time of simulation-based optimization of a reference trajectory. We find a 74% better-performing reference trajectory compared to nominal, and the numerical results clearly show a substantial reduction in computation time for designing future trajectories.
  arXiv  Detail & Related papers  (2023-03-30T15:44:30Z)
• Neural Posterior Estimation with Differentiable Simulators [58.720142291102135]
  We present a new method to perform Neural Posterior Estimation (NPE) with a differentiable simulator. We demonstrate how gradient information helps constrain the shape of the posterior and improves sample-efficiency.
  arXiv  Detail & Related papers  (2022-07-12T16:08:04Z)
• Synthetic Data-Based Simulators for Recommender Systems: A Survey [55.60116686945561]
  This survey aims at providing a comprehensive overview of the recent trends in the field of modeling and simulation. We start with the motivation behind the development of frameworks implementing the simulations -- simulators. We provide a new consistent classification of existing simulators based on their functionality, approbation, and industrial effectiveness.
  arXiv  Detail & Related papers  (2022-06-22T19:33:21Z)
• Truncated Marginal Neural Ratio Estimation [5.438798591410838]
  We present a neural simulator-based inference algorithm which simultaneously offers simulation efficiency and fast empirical posterior testability. Our approach is simulation efficient by simultaneously estimating low-dimensional marginal posteriors instead of the joint posterior. By estimating a locally amortized posterior our algorithm enables efficient empirical tests of the robustness of the inference results.
  arXiv  Detail & Related papers  (2021-07-02T18:00:03Z)
• Deep Bayesian Active Learning for Accelerating Stochastic Simulation [74.58219903138301]
  Interactive Neural Process (INP) is a deep active learning framework for simulations and with active learning approaches. For active learning, we propose a novel acquisition function, Latent Information Gain (LIG), calculated in the latent space of NP based models. The results demonstrate STNP outperforms the baselines in the learning setting and LIG achieves the state-of-the-art for active learning.
  arXiv  Detail & Related papers  (2021-06-05T01:31:51Z)
• Simulation-efficient marginal posterior estimation with swyft: stop wasting your precious time [5.533353383316288]
  We present algorithms for nested neural likelihood-to-evidence ratio estimation and simulation reuse. Together, these algorithms enable automatic and extremely simulator efficient estimation of marginal and joint posteriors.
  arXiv  Detail & Related papers  (2020-11-27T19:00:07Z)
• DISCO: Double Likelihood-free Inference Stochastic Control [29.84276469617019]
  We propose to leverage the power of modern simulators and recent techniques in Bayesian statistics for likelihood-free inference. The posterior distribution over simulation parameters is propagated through a potentially non-analytical model of the system. Experiments show that the controller proposed attained superior performance and robustness on classical control and robotics tasks.
  arXiv  Detail & Related papers  (2020-02-18T05:29:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.