Diffusion Schr\"odinger Bridge Matching

• URL: http://arxiv.org/abs/2303.16852v3
• Date: Tue, 12 Dec 2023 00:49:53 GMT
• Title: Diffusion Schr\"odinger Bridge Matching
• Authors: Yuyang Shi, Valentin De Bortoli, Andrew Campbell, Arnaud Doucet
• Abstract summary: We introduce Iterative Markovian Fitting (IMF) and Diffusion Schr"odinger Bridge Matching (DSBM) IMF is a new methodology for solving SB problems, and DSBM is a novel numerical algorithm for computing IMF iterates. We demonstrate the performance of DSBM on a variety of problems.
• Score: 36.95088080680221
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Solving transport problems, i.e. finding a map transporting one given distribution to another, has numerous applications in machine learning. Novel mass transport methods motivated by generative modeling have recently been proposed, e.g. Denoising Diffusion Models (DDMs) and Flow Matching Models (FMMs) implement such a transport through a Stochastic Differential Equation (SDE) or an Ordinary Differential Equation (ODE). However, while it is desirable in many applications to approximate the deterministic dynamic Optimal Transport (OT) map which admits attractive properties, DDMs and FMMs are not guaranteed to provide transports close to the OT map. In contrast, Schr\"odinger bridges (SBs) compute stochastic dynamic mappings which recover entropy-regularized versions of OT. Unfortunately, existing numerical methods approximating SBs either scale poorly with dimension or accumulate errors across iterations. In this work, we introduce Iterative Markovian Fitting (IMF), a new methodology for solving SB problems, and Diffusion Schr\"odinger Bridge Matching (DSBM), a novel numerical algorithm for computing IMF iterates. DSBM significantly improves over previous SB numerics and recovers as special/limiting cases various recent transport methods. We demonstrate the performance of DSBM on a variety of problems.

Related papers

• Schrödinger Bridge Flow for Unpaired Data Translation [38.19632736212184]
  We propose a novel algorithm to compute the Schr"odinger Bridge, a dynamic entropy-regularised version of Optimal Transport (OT) maps. This algorithm corresponds to a discretisation of a flow of path measures, which we call the Schr"odinger Bridge Flow. We demonstrate the performance of our algorithm on a variety of unpaired data translation tasks.
  arXiv  Detail & Related papers  (2024-09-14T07:34:30Z)
• Diffusion Bridge Implicit Models [25.213664260896103]
  Denoising diffusion bridge models (DDBMs) are a powerful variant of diffusion models for interpolating between two arbitrary paired distributions. We take the first step in fast sampling of DDBMs without extra training, motivated by the well-established recipes in diffusion models. We induce a novel, simple, and insightful form of ordinary differential equation (ODE) which inspires high-order numerical solvers.
  arXiv  Detail & Related papers  (2024-05-24T19:08:30Z)
• Adversarial Schrödinger Bridge Matching [66.39774923893103]
  Iterative Markovian Fitting (IMF) procedure alternates between Markovian and reciprocal projections of continuous-time processes. We propose a novel Discrete-time IMF (D-IMF) procedure in which learning of processes is replaced by learning just a few transition probabilities in discrete time. We show that our D-IMF procedure can provide the same quality of unpaired domain translation as the IMF, using only several generation steps instead of hundreds.
  arXiv  Detail & Related papers  (2024-05-23T11:29:33Z)
• Light and Optimal Schrödinger Bridge Matching [67.93806073192938]
  We propose a novel procedure to learn Schr"odinger Bridges (SB) which we call the textbf Schr"odinger bridge matching. We show that the optimal bridge matching objective coincides with the recently discovered energy-based modeling (EBM) objectives to learn EOT/SB. We develop a light solver (which we call LightSB-M) to implement optimal matching in practice using the mixture parameterization of the Schr"odinger potential.
  arXiv  Detail & Related papers  (2024-02-05T17:17:57Z)
• Gaussian Mixture Solvers for Diffusion Models [84.83349474361204]
  We introduce a novel class of SDE-based solvers called GMS for diffusion models. Our solver outperforms numerous SDE-based solvers in terms of sample quality in image generation and stroke-based synthesis.
  arXiv  Detail & Related papers  (2023-11-02T02:05:38Z)
• Denoising Diffusion Bridge Models [54.87947768074036]
  Diffusion models are powerful generative models that map noise to data using processes. For many applications such as image editing, the model input comes from a distribution that is not random noise. In our work, we propose Denoising Diffusion Bridge Models (DDBMs)
  arXiv  Detail & Related papers  (2023-09-29T03:24:24Z)
• Diffusion Bridge Mixture Transports, Schr\"odinger Bridge Problems and Generative Modeling [4.831663144935879]
  We propose a novel sampling-based iterative algorithm, the iterated diffusion bridge mixture (IDBM) procedure, aimed at solving the dynamic Schr"odinger bridge problem. The IDBM procedure exhibits the attractive property of realizing a valid transport between the target probability measures at each iteration.
  arXiv  Detail & Related papers  (2023-04-03T12:13:42Z)
• The Schr\"odinger Bridge between Gaussian Measures has a Closed Form [101.79851806388699]
  We focus on the dynamic formulation of OT, also known as the Schr"odinger bridge (SB) problem. In this paper, we provide closed-form expressions for SBs between Gaussian measures.
  arXiv  Detail & Related papers  (2022-02-11T15:59:01Z)
• Diffusion Schr\"odinger Bridge with Applications to Score-Based Generative Modeling [24.46142828617484]
  Diffusion SB is an original approximation of the Iterative Proportional Fitting (IPF) procedure to solve the Schr"odinger Bridge problem. We present Diffusion SB, an original approximation of the Iterative Proportional Fitting (IPF) procedure to solve the SB problem, and provide theoretical analysis along with generative modeling experiments.
  arXiv  Detail & Related papers  (2021-06-01T17:34:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.