Investigating the Impact of Model Misspecification in Neural Simulation-based Inference

• URL: http://arxiv.org/abs/2209.01845v1
• Date: Mon, 5 Sep 2022 09:08:16 GMT
• Title: Investigating the Impact of Model Misspecification in Neural Simulation-based Inference
• Authors: Patrick Cannon, Daniel Ward, Sebastian M. Schmon
• Abstract summary: We study the behaviour of neural SBI algorithms in the presence of various forms of model misspecification. We find that misspecification can have a profoundly deleterious effect on performance. We conclude that new approaches are required to address model misspecification if neural SBI algorithms are to be relied upon to derive accurate conclusions.
• Score: 1.933681537640272
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Aided by advances in neural density estimation, considerable progress has been made in recent years towards a suite of simulation-based inference (SBI) methods capable of performing flexible, black-box, approximate Bayesian inference for stochastic simulation models. While it has been demonstrated that neural SBI methods can provide accurate posterior approximations, the simulation studies establishing these results have considered only well-specified problems -- that is, where the model and the data generating process coincide exactly. However, the behaviour of such algorithms in the case of model misspecification has received little attention. In this work, we provide the first comprehensive study of the behaviour of neural SBI algorithms in the presence of various forms of model misspecification. We find that misspecification can have a profoundly deleterious effect on performance. Some mitigation strategies are explored, but no approach tested prevents failure in all cases. We conclude that new approaches are required to address model misspecification if neural SBI algorithms are to be relied upon to derive accurate scientific conclusions.

Related papers

• Supervised Score-Based Modeling by Gradient Boosting [49.556736252628745]
  We propose a Supervised Score-based Model (SSM) which can be viewed as a gradient boosting algorithm combining score matching. We provide a theoretical analysis of learning and sampling for SSM to balance inference time and prediction accuracy. Our model outperforms existing models in both accuracy and inference time.
  arXiv  Detail & Related papers  (2024-11-02T07:06:53Z)
• Detecting Model Misspecification in Amortized Bayesian Inference with Neural Networks: An Extended Investigation [9.950524371154394]
  We propose a new misspecification measure that can be trained in an unsupervised fashion and reliably detects model misspecification at test time. We show how the proposed misspecification test warns users about suspicious outputs, raises an alarm when predictions are not trustworthy, and guides model designers in their search for better simulators.
  arXiv  Detail & Related papers  (2024-06-05T11:30:16Z)
• Preconditioned Neural Posterior Estimation for Likelihood-free Inference [5.651060979874024]
  We show in this paper that the neural posterior estimator (NPE) methods are not guaranteed to be highly accurate, even on problems with low dimension. We propose preconditioned NPE and its sequential version (PSNPE), which uses a short run of ABC to effectively eliminate regions of parameter space that produce large discrepancy between simulations and data. We present comprehensive empirical evidence that this melding of neural and statistical SBI methods improves performance over a range of examples.
  arXiv  Detail & Related papers  (2024-04-21T07:05:38Z)
• Calibrating Neural Simulation-Based Inference with Differentiable Coverage Probability [50.44439018155837]
  We propose to include a calibration term directly into the training objective of the neural model. By introducing a relaxation of the classical formulation of calibration error we enable end-to-end backpropagation. It is directly applicable to existing computational pipelines allowing reliable black-box posterior inference.
  arXiv  Detail & Related papers  (2023-10-20T10:20:45Z)
• Learning Robust Statistics for Simulation-based Inference under Model Misspecification [23.331522354991527]
  We propose the first general approach to handle model misspecification that works across different classes of simulation-based inference methods. We show that our method yields robust inference in misspecified scenarios, whilst still being accurate when the model is well-specified.
  arXiv  Detail & Related papers  (2023-05-25T09:06:26Z)
• Misspecification-robust Sequential Neural Likelihood for Simulation-based Inference [0.20971479389679337]
  We propose a novel SNL method, which through the incorporation of additional adjustment parameters, is robust to model misspecification. We demonstrate the efficacy of our approach through several illustrative examples.
  arXiv  Detail & Related papers  (2023-01-31T02:28:18Z)
• Robust Neural Posterior Estimation and Statistical Model Criticism [1.5749416770494706]
  We argue that modellers must treat simulators as idealistic representations of the true data generating process. In this work we revisit neural posterior estimation (NPE), a class of algorithms that enable black-box parameter inference in simulation models. We find that the presence of misspecification, in contrast, leads to unreliable inference when NPE is used naively.
  arXiv  Detail & Related papers  (2022-10-12T20:06:55Z)
• Neural Causal Models for Counterfactual Identification and Estimation [62.30444687707919]
  We study the evaluation of counterfactual statements through neural models. First, we show that neural causal models (NCMs) are expressive enough. Second, we develop an algorithm for simultaneously identifying and estimating counterfactual distributions.
  arXiv  Detail & Related papers  (2022-09-30T18:29:09Z)
• MissDAG: Causal Discovery in the Presence of Missing Data with Continuous Additive Noise Models [78.72682320019737]
  We develop a general method, which we call MissDAG, to perform causal discovery from data with incomplete observations. MissDAG maximizes the expected likelihood of the visible part of observations under the expectation-maximization framework. We demonstrate the flexibility of MissDAG for incorporating various causal discovery algorithms and its efficacy through extensive simulations and real data experiments.
  arXiv  Detail & Related papers  (2022-05-27T09:59:46Z)
• Estimation of Bivariate Structural Causal Models by Variational Gaussian Process Regression Under Likelihoods Parametrised by Normalising Flows [74.85071867225533]
  Causal mechanisms can be described by structural causal models. One major drawback of state-of-the-art artificial intelligence is its lack of explainability.
  arXiv  Detail & Related papers  (2021-09-06T14:52:58Z)
• The Causal Neural Connection: Expressiveness, Learnability, and Inference [125.57815987218756]
  An object called structural causal model (SCM) represents a collection of mechanisms and sources of random variation of the system under investigation. In this paper, we show that the causal hierarchy theorem (Thm. 1, Bareinboim et al., 2020) still holds for neural models. We introduce a special type of SCM called a neural causal model (NCM), and formalize a new type of inductive bias to encode structural constraints necessary for performing causal inferences.
  arXiv  Detail & Related papers  (2021-07-02T01:55:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.