Flow-Transformed Implicit Processes for Function-Space Variational Inference

• URL: http://arxiv.org/abs/2606.01954v1
• Date: Mon, 01 Jun 2026 09:14:09 GMT
• Title: Flow-Transformed Implicit Processes for Function-Space Variational Inference
• Abstract summary: Implicit-process priors define distributions over functions through flexible generative mechanisms.<n>One practical strategy is to approximate the prior using a finite collection of sampled functions, and then represent posterior functions as learned combinations of these samples.<n>We propose Flow-Transformed Implicit Processes (FTIP), a variational inference method that makes this finite-dimensional function-space approximation more expressive.
• Score: 4.970364068620607
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Implicit-process priors define distributions over functions through flexible generative mechanisms, making them attractive for Bayesian function-space modelling. However, performing posterior inference with such priors is challenging because their induced function-space distributions are typically not available in closed form. One practical strategy is to approximate the prior using a finite collection of sampled functions, and then represent posterior functions as learned combinations of these samples. Existing approaches commonly place a Gaussian variational distribution over the combination weights. While tractable, this choice limits the shapes of posterior uncertainty that can be represented, especially when the true posterior is asymmetric, heavy-tailed, or multimodal. We propose Flow-Transformed Implicit Processes (FTIP), a variational inference method that makes this finite-dimensional function-space approximation more expressive. Instead of using a Gaussian distribution over the combination weights, FTIP uses a normalizing flow to define a richer variational distribution. This induces a flexible posterior distribution over functions while preserving tractable optimization. We train the model using a Black-Box α objective, allowing us to compare mass-covering and mode-seeking variational behaviour. Experiments show that FTIP captures asymmetric and multimodal posterior structure in function space that Gaussian coefficient approximations tend to smooth or collapse.

Related papers

• Amortized Variational Inference for Joint Posterior and Predictive Distributions in Bayesian Uncertainty Quantification [1.0742675209112622]
  We propose a variational Bayesian framework that directly targets the posterior-predictive distribution and jointly learns variational approximations of both the posterior and the corresponding predictive distribution.<n>The proposed method achieves more accurate predictive distributions than conventional two-stage variational inference, while substantially reducing the cost of online predictive inference.
  arXiv  Detail & Related papers  (2026-05-05T12:56:00Z)
• Supervised Guidance Training for Infinite-Dimensional Diffusion Models [47.65586147952857]
  In inverse problems, the aim is to sample from a posterior distribution over functions obtained by conditioning a prior.<n>We prove that the models can be conditioned using an infinite-dimensional extension of Doob's $h$-transform.<n>We propose a simulation-free score matching objective (called Supervised Guidance Training) enabling efficient and stable posterior sampling.
  arXiv  Detail & Related papers  (2026-01-28T16:39:39Z)
• An Elementary Approach to Scheduling in Generative Diffusion Models [55.171367482496755]
  An elementary approach to characterizing the impact of noise scheduling and time discretization in generative diffusion models is developed.<n> Experiments across different datasets and pretrained models demonstrate that the time discretization strategy selected by our approach consistently outperforms baseline and search-based strategies.
  arXiv  Detail & Related papers  (2026-01-20T05:06:26Z)
• Sequential Function-Space Variational Inference via Gaussian Mixture Approximation [0.6827423171182154]
  Continual learning in neural networks aims to learn new tasks without forgetting old tasks.<n>We propose an SFSVI method based on a Gaussian mixture variational distribution.<n>We find that in terms of final average accuracy, likelihood-focused Gaussian mixture SFSVI outperforms other sequential variational inference methods.
  arXiv  Detail & Related papers  (2025-03-10T09:38:35Z)
• Improving Diffusion Models for Inverse Problems Using Optimal Posterior Covariance [52.093434664236014]
  Recent diffusion models provide a promising zero-shot solution to noisy linear inverse problems without retraining for specific inverse problems. Inspired by this finding, we propose to improve recent methods by using more principled covariance determined by maximum likelihood estimation.
  arXiv  Detail & Related papers  (2024-02-03T13:35:39Z)
• Diffusion models for probabilistic programming [56.47577824219207]
  Diffusion Model Variational Inference (DMVI) is a novel method for automated approximate inference in probabilistic programming languages (PPLs) DMVI is easy to implement, allows hassle-free inference in PPLs without the drawbacks of, e.g., variational inference using normalizing flows, and does not make any constraints on the underlying neural network model.
  arXiv  Detail & Related papers  (2023-11-01T12:17:05Z)
• Noise-Free Sampling Algorithms via Regularized Wasserstein Proximals [3.4240632942024685]
  We consider the problem of sampling from a distribution governed by a potential function. This work proposes an explicit score based MCMC method that is deterministic, resulting in a deterministic evolution for particles.
  arXiv  Detail & Related papers  (2023-08-28T23:51:33Z)
• Robust scalable initialization for Bayesian variational inference with multi-modal Laplace approximations [0.0]
  Variational mixtures with full-covariance structures suffer from a quadratic growth due to variational parameters with the number of parameters. We propose a method for constructing an initial Gaussian model approximation that can be used to warm-start variational inference.
  arXiv  Detail & Related papers  (2023-07-12T19:30:04Z)
• Neural Diffusion Processes [12.744250155946503]
  We propose Neural Diffusion Processes (NDPs), a novel approach that learns to sample from a rich distribution over functions through its finite marginals. We empirically show that NDPs can capture functional distributions close to the true Bayesian posterior. NDPs enable a variety of downstream tasks, including regression, implicit hyper marginalisation, non-Gaussian posterior prediction and global optimisation.
  arXiv  Detail & Related papers  (2022-06-08T16:13:04Z)
• Efficient CDF Approximations for Normalizing Flows [64.60846767084877]
  We build upon the diffeomorphic properties of normalizing flows to estimate the cumulative distribution function (CDF) over a closed region. Our experiments on popular flow architectures and UCI datasets show a marked improvement in sample efficiency as compared to traditional estimators.
  arXiv  Detail & Related papers  (2022-02-23T06:11:49Z)
• Non-Gaussian Gaussian Processes for Few-Shot Regression [71.33730039795921]
  We propose an invertible ODE-based mapping that operates on each component of the random variable vectors and shares the parameters across all of them. NGGPs outperform the competing state-of-the-art approaches on a diversified set of benchmarks and applications.
  arXiv  Detail & Related papers  (2021-10-26T10:45:25Z)
• Dimension-robust Function Space MCMC With Neural Network Priors [0.0]
  This paper introduces a new prior on functions spaces which scales more favourably in the dimension of the function's domain. We show that our resulting posterior of the unknown function is amenable to sampling using Hilbert space Markov chain Monte Carlo methods. We show that our priors are competitive and have distinct advantages over other function space priors.
  arXiv  Detail & Related papers  (2020-12-20T14:52:57Z)
• Gaussianization Flows [113.79542218282282]
  We propose a new type of normalizing flow model that enables both efficient iteration of likelihoods and efficient inversion for sample generation. Because of this guaranteed expressivity, they can capture multimodal target distributions without compromising the efficiency of sample generation.
  arXiv  Detail & Related papers  (2020-03-04T08:15:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.