Model Informed Flows for Bayesian Inference of Probabilistic Programs

• URL: http://arxiv.org/abs/2505.24243v1
• Date: Fri, 30 May 2025 06:08:00 GMT
• Title: Model Informed Flows for Bayesian Inference of Probabilistic Programs
• Authors: Joohwan Ko, Justin Domke,
• Abstract summary: We show that VIP and a full-rank Gaussian can be represented exactly as a forward autoregressive flow augmented with a translation term and input from the model's prior.<n>We introduce the Model-Informed Flow architecture, which adds the necessary translation mechanism, prior information, and hierarchical ordering.
• Score: 18.937801725778538
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Variational inference often struggles with the posterior geometry exhibited by complex hierarchical Bayesian models. Recent advances in flow-based variational families and Variationally Inferred Parameters (VIP) each address aspects of this challenge, but their formal relationship is unexplored. Here, we prove that the combination of VIP and a full-rank Gaussian can be represented exactly as a forward autoregressive flow augmented with a translation term and input from the model's prior. Guided by this theoretical insight, we introduce the Model-Informed Flow (MIF) architecture, which adds the necessary translation mechanism, prior information, and hierarchical ordering. Empirically, MIF delivers tighter posterior approximations and matches or exceeds state-of-the-art performance across a suite of hierarchical and non-hierarchical benchmarks.

Related papers

• Amortized variational transdimensional inference [8.890385058609867]
  CoSMIC is an extension to neural autoregressive conditional normalizing flow architectures.<n>We propose a combined variational transdimensional inference (VTI) approach to training CoSMIC flows.
  arXiv  Detail & Related papers  (2025-06-05T08:33:07Z)
• FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocities [76.46448367752944]
  multimodal large language models (MLLMs) unify visual understanding and image generation within a single framework.<n>Most existing MLLMs rely on autore (AR) architectures, which impose inherent limitations on future development.<n>We introduce FUDOKI, a unified multimodal model purely based on discrete flow matching.
  arXiv  Detail & Related papers  (2025-05-26T15:46:53Z)
• HOFAR: High-Order Augmentation of Flow Autoregressive Transformers [17.002793355495136]
  This paper introduces a novel framework that systematically enhances flow autoregressive transformers through high-order supervision.<n>We provide theoretical analysis and empirical evaluation showing that our High-Order FlowAR (HOFAR) demonstrates measurable improvements in generation quality compared to baseline models.
  arXiv  Detail & Related papers  (2025-03-11T04:29:22Z)
• Preconditioned Inexact Stochastic ADMM for Deep Model [35.37705488695026]
  This paper develops an algorithm, PISA, which enables scalable parallel computing and supports various second-moment schemes.<n>Grounded in rigorous theoretical guarantees, the algorithm converges under the sole assumption of Lipschitz of the gradient.<n> Comprehensive experimental evaluations for or fine-tuning diverse FMs, including vision models, large language models, reinforcement learning models, generative adversarial networks, and recurrent neural networks, demonstrate its superior numerical performance compared to various state-of-the-art Directions.
  arXiv  Detail & Related papers  (2025-02-15T12:28:51Z)
• Exchangeable Sequence Models Quantify Uncertainty Over Latent Concepts [6.256239986541708]
  We show that pre-trained sequence models are naturally capable of probabilistic reasoning over exchangeable data points.<n>A sequence model learns the relationship between observations, which differs from typical Bayesian models.<n>We show the sequence prediction loss controls the quality of uncertainty quantification.
  arXiv  Detail & Related papers  (2024-08-06T17:16:10Z)
• Variational Laplace Autoencoders [53.08170674326728]
  Variational autoencoders employ an amortized inference model to approximate the posterior of latent variables. We present a novel approach that addresses the limited posterior expressiveness of fully-factorized Gaussian assumption. We also present a general framework named Variational Laplace Autoencoders (VLAEs) for training deep generative models.
  arXiv  Detail & Related papers  (2022-11-30T18:59:27Z)
• A Variational Inference Approach to Inverse Problems with Gamma Hyperpriors [60.489902135153415]
  This paper introduces a variational iterative alternating scheme for hierarchical inverse problems with gamma hyperpriors. The proposed variational inference approach yields accurate reconstruction, provides meaningful uncertainty quantification, and is easy to implement.
  arXiv  Detail & Related papers  (2021-11-26T06:33:29Z)
• Embedded-model flows: Combining the inductive biases of model-free deep learning and explicit probabilistic modeling [8.405013085269976]
  We propose embedded-model flows which alternate general-purpose transformations with structured layers that embed domain-specific inductive biases. We demonstrate that EMFs can be used to induce desirable properties such as multimodality, hierarchical coupling and continuity. In experiments, we show that this approach outperforms state-of-the-art methods in common structured inference problems.
  arXiv  Detail & Related papers  (2021-10-12T14:12:16Z)
• Post-mortem on a deep learning contest: a Simpson's paradox and the complementary roles of scale metrics versus shape metrics [61.49826776409194]
  We analyze a corpus of models made publicly-available for a contest to predict the generalization accuracy of neural network (NN) models. We identify what amounts to a Simpson's paradox: where "scale" metrics perform well overall but perform poorly on sub partitions of the data. We present two novel shape metrics, one data-independent, and the other data-dependent, which can predict trends in the test accuracy of a series of NNs.
  arXiv  Detail & Related papers  (2021-06-01T19:19:49Z)
• Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modeling [54.94763543386523]
  Current autoencoder-based disentangled representation learning methods achieve disentanglement by penalizing the ( aggregate) posterior to encourage statistical independence of the latent factors. We present a novel multi-stage modeling approach where the disentangled factors are first learned using a penalty-based disentangled representation learning method. Then, the low-quality reconstruction is improved with another deep generative model that is trained to model the missing correlated latent variables.
  arXiv  Detail & Related papers  (2020-10-25T18:51:15Z)
• Posterior Differential Regularization with f-divergence for Improving Model Robustness [95.05725916287376]
  We focus on methods that regularize the model posterior difference between clean and noisy inputs. We generalize the posterior differential regularization to the family of $f$-divergences. Our experiments show that regularizing the posterior differential with $f$-divergence can result in well-improved model robustness.
  arXiv  Detail & Related papers  (2020-10-23T19:58:01Z)
• Self-Reflective Variational Autoencoder [21.054722609128525]
  Variational Autoencoder (VAE) is a powerful framework for learning latent variable generative models. We introduce a solution, which we call self-reflective inference. We empirically demonstrate the clear advantages of matching the variational posterior to the exact posterior.
  arXiv  Detail & Related papers  (2020-07-10T05:05:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.