Bayesian Inverse Problems Meet Flow Matching: Efficient and Flexible Inference via Transformers

• URL: http://arxiv.org/abs/2503.01375v2
• Date: Fri, 16 May 2025 19:52:08 GMT
• Title: Bayesian Inverse Problems Meet Flow Matching: Efficient and Flexible Inference via Transformers
• Authors: Daniil Sherki, Ivan Oseledets, Ekaterina Muravleva,
• Abstract summary: We propose a novel framework that integrates Conditional Flow Matching with a transformer-based architecture to enable fast and flexible sampling from complex posterior distributions.<n>The efficacy of the proposed framework is demonstrated through its application to three problems: a simple nonlinear model, a disease dynamics framework, and a two-dimensional Darcy flow Partial Differential Equation.
• Score: 3.868222899558346
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The efficient resolution of Bayesian inverse problems remains challenging due to the high computational cost of traditional sampling methods. In this paper, we propose a novel framework that integrates Conditional Flow Matching (CFM) with a transformer-based architecture to enable fast and flexible sampling from complex posterior distributions. The proposed methodology involves the direct learning of conditional probability trajectories from the data, leveraging CFM's ability to bypass iterative simulation and transformers' capacity to process arbitrary numbers of observations. The efficacy of the proposed framework is demonstrated through its application to three problems: a simple nonlinear model, a disease dynamics framework, and a two-dimensional Darcy flow Partial Differential Equation. The primary outcomes demonstrate that the relative errors in parameters recovery are as low as 1.5%, and that the inference time is reduced by up to 2000 times on CPU in comparison with the Monte Carlo Markov Chain. This framework facilitates the expeditious resolution of Bayesian problems through the utilisation of sampling from the learned conditional distribution.

Related papers

• Sequential Bayesian Design for Efficient Surrogate Construction in the Inversion of Darcy Flows [2.8222764383589456]
  Inverse problems governed by partial differential equations (PDEs) play a crucial role in various fields, including computational science, image processing, and engineering.<n>We introduce a sequential Bayesian design strategy to acquire the proposed surrogate since the high-probability region of the likelihood is unknown.<n>Three experiments based the Darcy flow equation demonstrate the advantages of the proposed method in terms of both inversion accuracy and computational speed.
  arXiv  Detail & Related papers  (2025-07-23T17:25:14Z)
• Diffusion Models for Solving Inverse Problems via Posterior Sampling with Piecewise Guidance [52.705112811734566]
  A novel diffusion-based framework is introduced for solving inverse problems using a piecewise guidance scheme.<n>The proposed method is problem-agnostic and readily adaptable to a variety of inverse problems.<n>The framework achieves a reduction in inference time of (25%) for inpainting with both random and center masks, and (23%) and (24%) for (4times) and (8times) super-resolution tasks.
  arXiv  Detail & Related papers  (2025-07-22T19:35:14Z)
• Variational Autoencoder for Generating Broader-Spectrum prior Proposals in Markov chain Monte Carlo Methods [0.0]
  This study uses a Variational Autoencoder method to enhance the efficiency and applicability of Markov Chain Monte Carlo (McMC) methods.<n>The VAE framework enables a data-driven approach to flexibly capture a broader range of correlation structures in inverse problems.
  arXiv  Detail & Related papers  (2025-06-16T14:11:16Z)
• Solving Inverse Problems with FLAIR [59.02385492199431]
  Flow-based latent generative models are able to generate images with remarkable quality, even enabling text-to-image generation.<n>We present FLAIR, a novel training free variational framework that leverages flow-based generative models as a prior for inverse problems.<n>Results on standard imaging benchmarks demonstrate that FLAIR consistently outperforms existing diffusion- and flow-based methods in terms of reconstruction quality and sample diversity.
  arXiv  Detail & Related papers  (2025-06-03T09:29:47Z)
• Discrete Neural Flow Samplers with Locally Equivariant Transformer [25.911046280803586]
  We propose Discrete Neural Flow Samplers (DNFS), a trainable and efficient framework for discrete sampling.<n>DNFS learns the rate matrix of a continuous-time Markov chain such that the resulting dynamics satisfy the Kolmogorov equation.<n>To further facilitate computational efficiency, we propose locally equivaraint Transformer, a novel parameterisation of the rate matrix.
  arXiv  Detail & Related papers  (2025-05-23T11:06:06Z)
• Solving High-dimensional Inverse Problems Using Amortized Likelihood-free Inference with Noisy and Incomplete Data [43.43717668587333]
  We present a likelihood-free probabilistic inversion method based on normalizing flows for high-dimensional inverse problems.<n>The proposed method is composed of two complementary networks: a summary network for data compression and an inference network for parameter estimation.<n>We apply the proposed method to an inversion problem in groundwater hydrology to estimate the posterior distribution of the log-conductivity field conditioned on spatially sparse time-series observations.
  arXiv  Detail & Related papers  (2024-12-05T19:13:17Z)
• On the Wasserstein Convergence and Straightness of Rectified Flow [54.580605276017096]
  Rectified Flow (RF) is a generative model that aims to learn straight flow trajectories from noise to data.<n>We provide a theoretical analysis of the Wasserstein distance between the sampling distribution of RF and the target distribution.<n>We present general conditions guaranteeing uniqueness and straightness of 1-RF, which is in line with previous empirical findings.
  arXiv  Detail & Related papers  (2024-10-19T02:36:11Z)
• Unveiling the Statistical Foundations of Chain-of-Thought Prompting Methods [59.779795063072655]
  Chain-of-Thought (CoT) prompting and its variants have gained popularity as effective methods for solving multi-step reasoning problems. We analyze CoT prompting from a statistical estimation perspective, providing a comprehensive characterization of its sample complexity.
  arXiv  Detail & Related papers  (2024-08-25T04:07:18Z)
• Amortized Posterior Sampling with Diffusion Prior Distillation [55.03585818289934]
  Amortized Posterior Sampling is a novel variational inference approach for efficient posterior sampling in inverse problems.<n>Our method trains a conditional flow model to minimize the divergence between the variational distribution and the posterior distribution implicitly defined by the diffusion model.<n>Unlike existing methods, our approach is unsupervised, requires no paired training data, and is applicable to both Euclidean and non-Euclidean domains.
  arXiv  Detail & Related papers  (2024-07-25T09:53:12Z)
• Efficient, Multimodal, and Derivative-Free Bayesian Inference With Fisher-Rao Gradient Flows [10.153270126742369]
  We study efficient approximate sampling for probability distributions known up to normalization constants. We specifically focus on a problem class arising in Bayesian inference for large-scale inverse problems in science and engineering applications.
  arXiv  Detail & Related papers  (2024-06-25T04:07:22Z)
• Scalable diffusion posterior sampling in infinite-dimensional inverse problems [5.340736751238338]
  We propose a scalable diffusion posterior sampling (SDPS) method to bypass forward mapping evaluations during sampling. The approach is shown to generalize to infinite-dimensional diffusion models and is validated through rigorous convergence analysis and high-dimensional CT imaging experiments.
  arXiv  Detail & Related papers  (2024-05-24T15:33:27Z)
• A Variational Perspective on Solving Inverse Problems with Diffusion Models [101.831766524264]
  Inverse tasks can be formulated as inferring a posterior distribution over data. This is however challenging in diffusion models since the nonlinear and iterative nature of the diffusion process renders the posterior intractable. We propose a variational approach that by design seeks to approximate the true posterior distribution.
  arXiv  Detail & Related papers  (2023-05-07T23:00:47Z)
• Inverse Models for Estimating the Initial Condition of Spatio-Temporal Advection-Diffusion Processes [5.814371485767541]
  Inverse problems involve making inference about unknown parameters of a physical process using observational data. This paper investigates the estimation of the initial condition of a-temporal advection-diffusion process using spatially sparse data streams.
  arXiv  Detail & Related papers  (2023-02-08T15:30:16Z)
• Diffusion Posterior Sampling for General Noisy Inverse Problems [50.873313752797124]
  We extend diffusion solvers to handle noisy (non)linear inverse problems via approximation of the posterior sampling. Our method demonstrates that diffusion models can incorporate various measurement noise statistics.
  arXiv  Detail & Related papers  (2022-09-29T11:12:27Z)
• Conditioning Normalizing Flows for Rare Event Sampling [61.005334495264194]
  We propose a transition path sampling scheme based on neural-network generated configurations. We show that this approach enables the resolution of both the thermodynamics and kinetics of the transition region.
  arXiv  Detail & Related papers  (2022-07-29T07:56:10Z)
• Improving Diffusion Models for Inverse Problems using Manifold Constraints [55.91148172752894]
  We show that current solvers throw the sample path off the data manifold, and hence the error accumulates. To address this, we propose an additional correction term inspired by the manifold constraint. We show that our method is superior to the previous methods both theoretically and empirically.
  arXiv  Detail & Related papers  (2022-06-02T09:06:10Z)
• Efficient CDF Approximations for Normalizing Flows [64.60846767084877]
  We build upon the diffeomorphic properties of normalizing flows to estimate the cumulative distribution function (CDF) over a closed region. Our experiments on popular flow architectures and UCI datasets show a marked improvement in sample efficiency as compared to traditional estimators.
  arXiv  Detail & Related papers  (2022-02-23T06:11:49Z)
• Stochastic Normalizing Flows for Inverse Problems: a Markov Chains Viewpoint [0.45119235878273]
  We consider normalizing flows from a Markov chain point of view. We replace transition densities by general Markov kernels and establish proofs via Radon-Nikodym derivatives. The performance of the proposed conditional normalizing flow is demonstrated by numerical examples.
  arXiv  Detail & Related papers  (2021-09-23T13:44:36Z)
• A variational inference framework for inverse problems [0.39373541926236766]
  A framework is presented for fitting inverse problem models via variational Bayes approximations. This methodology guarantees flexibility to statistical model specification for a broad range of applications. An image processing application and a simulation exercise motivated by biomedical problems reveal the computational advantage offered by variational Bayes.
  arXiv  Detail & Related papers  (2021-03-10T07:37:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.