Deep conditional distribution learning via conditional Föllmer flow

• URL: http://arxiv.org/abs/2402.01460v2
• Date: Thu, 13 Jun 2024 08:11:10 GMT
• Title: Deep conditional distribution learning via conditional Föllmer flow
• Authors: Jinyuan Chang, Zhao Ding, Yuling Jiao, Ruoxuan Li, Jerry Zhijian Yang,
• Abstract summary: We introduce an ordinary differential equation (ODE) based deep generative method for learning conditional distributions, named Conditional F"ollmer Flow. For effective implementation, we discretize the flow with Euler's method where we estimate the velocity field nonparametrically using a deep neural network.
• Score: 3.227277661633986
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce an ordinary differential equation (ODE) based deep generative method for learning conditional distributions, named Conditional F\"ollmer Flow. Starting from a standard Gaussian distribution, the proposed flow could approximate the target conditional distribution very well when the time is close to 1. For effective implementation, we discretize the flow with Euler's method where we estimate the velocity field nonparametrically using a deep neural network. Furthermore, we also establish the convergence result for the Wasserstein-2 distance between the distribution of the learned samples and the target conditional distribution, providing the first comprehensive end-to-end error analysis for conditional distribution learning via ODE flow. Our numerical experiments showcase its effectiveness across a range of scenarios, from standard nonparametric conditional density estimation problems to more intricate challenges involving image data, illustrating its superiority over various existing conditional density estimation methods.

Related papers

• Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplers [49.97755400231656]
  We present the first performance guarantee with explicit dimensional general score-mismatched diffusion samplers. We show that score mismatches result in an distributional bias between the target and sampling distributions, proportional to the accumulated mismatch between the target and training distributions. This result can be directly applied to zero-shot conditional samplers for any conditional model, irrespective of measurement noise.
  arXiv  Detail & Related papers  (2024-10-17T16:42:12Z)
• Schr\"odinger bridge based deep conditional generative learning [0.0]
  We introduce a novel Schr"odinger bridge based deep generative method for learning conditional distributions. We apply our method to both low-dimensional and high-dimensional conditional generation problems.
  arXiv  Detail & Related papers  (2024-09-25T19:08:13Z)
• Flow matching achieves almost minimax optimal convergence [50.38891696297888]
  Flow matching (FM) has gained significant attention as a simulation-free generative model. This paper discusses the convergence properties of FM for large sample size under the $p$-Wasserstein distance. We establish that FM can achieve an almost minimax optimal convergence rate for $1 leq p leq 2$, presenting the first theoretical evidence that FM can reach convergence rates comparable to those of diffusion models.
  arXiv  Detail & Related papers  (2024-05-31T14:54:51Z)
• Distributed Markov Chain Monte Carlo Sampling based on the Alternating Direction Method of Multipliers [143.6249073384419]
  In this paper, we propose a distributed sampling scheme based on the alternating direction method of multipliers. We provide both theoretical guarantees of our algorithm's convergence and experimental evidence of its superiority to the state-of-the-art. In simulation, we deploy our algorithm on linear and logistic regression tasks and illustrate its fast convergence compared to existing gradient-based methods.
  arXiv  Detail & Related papers  (2024-01-29T02:08:40Z)
• Conditional Stochastic Interpolation for Generative Learning [5.0061421661196865]
  We propose a conditional conditional (CSI) method for learning conditional distributions. We derive explicit expressions of the conditional drift and score functions in terms of conditional expectations. We illustrate the application of CSI on image generation using a benchmark image dataset.
  arXiv  Detail & Related papers  (2023-12-09T13:53:35Z)
• Sobolev Space Regularised Pre Density Models [51.558848491038916]
  We propose a new approach to non-parametric density estimation that is based on regularizing a Sobolev norm of the density. This method is statistically consistent, and makes the inductive validation model clear and consistent.
  arXiv  Detail & Related papers  (2023-07-25T18:47:53Z)
• Error Bounds for Flow Matching Methods [38.9898500163582]
  Flow matching methods approximate a flow between two arbitrary probability distributions. We present error bounds for the flow matching procedure using fully deterministic sampling, assuming an $L2$ bound on the approximation error and a certain regularity on the data distributions.
  arXiv  Detail & Related papers  (2023-05-26T12:13:53Z)
• Building Normalizing Flows with Stochastic Interpolants [11.22149158986164]
  A simple generative quadratic model based on a continuous-time normalizing flow between any pair of base and target distributions is proposed. The velocity field of this flow is inferred from the probability current of a time-dependent distribution that interpolates between the base and the target in finite time.
  arXiv  Detail & Related papers  (2022-09-30T16:30:31Z)
• Efficient CDF Approximations for Normalizing Flows [64.60846767084877]
  We build upon the diffeomorphic properties of normalizing flows to estimate the cumulative distribution function (CDF) over a closed region. Our experiments on popular flow architectures and UCI datasets show a marked improvement in sample efficiency as compared to traditional estimators.
  arXiv  Detail & Related papers  (2022-02-23T06:11:49Z)
• Composing Normalizing Flows for Inverse Problems [89.06155049265641]
  We propose a framework for approximate inference that estimates the target conditional as a composition of two flow models. Our method is evaluated on a variety of inverse problems and is shown to produce high-quality samples with uncertainty.
  arXiv  Detail & Related papers  (2020-02-26T19:01:11Z)
• Learning Implicit Generative Models with Theoretical Guarantees [12.761710596142109]
  We propose a textbfunified textbfframework for textbfimplicit textbfmodeling (UnifiGem) UnifiGem integrates approaches from optimal transport, numerical ODE, density-ratio (density-difference) estimation and deep neural networks. Experimental results on both synthetic datasets and real benchmark datasets support our theoretical findings and demonstrate the effectiveness of UnifiGem.
  arXiv  Detail & Related papers  (2020-02-07T15:55:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.