Comparing Probability Distributions with Conditional Transport

• URL: http://arxiv.org/abs/2012.14100v3
• Date: Fri, 9 Apr 2021 02:00:37 GMT
• Title: Comparing Probability Distributions with Conditional Transport
• Authors: Huangjie Zheng and Mingyuan Zhou
• Abstract summary: We propose conditional transport (CT) as a new divergence and approximate it with the amortized CT (ACT) cost. ACT amortizes the computation of its conditional transport plans and comes with unbiased sample gradients that are straightforward to compute. On a wide variety of benchmark datasets generative modeling, substituting the default statistical distance of an existing generative adversarial network with ACT is shown to consistently improve the performance.
• Score: 63.11403041984197
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To measure the difference between two probability distributions, we propose conditional transport (CT) as a new divergence and further approximate it with the amortized CT (ACT) cost to make it amenable to implicit distributions and stochastic gradient descent based optimization. ACT amortizes the computation of its conditional transport plans and comes with unbiased sample gradients that are straightforward to compute. When applied to train a generative model, ACT is shown to strike a good balance between mode covering and seeking behaviors and strongly resist mode collapse. On a wide variety of benchmark datasets for generative modeling, substituting the default statistical distance of an existing generative adversarial network with ACT is shown to consistently improve the performance.

Related papers

• Minibatch Optimal Transport and Perplexity Bound Estimation in Discrete Flow Matching [0.0]
  Outperforming autoregressive models on categorical data distributions, such as textual data, remains challenging for continuous diffusion and flow models. We propose a dynamic-optimal-transport-like minimization objective for discrete flows with convex interpolants. Unlike continuous flows, wherein the instantaneous change of variables enables density estimation, discrete models lack a similar mechanism due to the inherent non-determinism and discontinuity of their paths.
  arXiv  Detail & Related papers  (2024-11-01T17:35:09Z)
• Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplers [49.97755400231656]
  We present the first performance guarantee with explicit dimensional general score-mismatched diffusion samplers. We show that score mismatches result in an distributional bias between the target and sampling distributions, proportional to the accumulated mismatch between the target and training distributions. This result can be directly applied to zero-shot conditional samplers for any conditional model, irrespective of measurement noise.
  arXiv  Detail & Related papers  (2024-10-17T16:42:12Z)
• Variational Schrödinger Diffusion Models [14.480273869571468]
  Schr"odinger bridge (SB) has emerged as the go-to method for optimizing transportation plans in diffusion models. We leverage variational inference to linearize the forward score functions (variational scores) of SB. We propose the variational Schr"odinger diffusion model (VSDM), where the forward process is a multivariate diffusion and the variational scores are adaptively optimized for efficient transport.
  arXiv  Detail & Related papers  (2024-05-08T04:01:40Z)
• Optimal Transport-Guided Conditional Score-Based Diffusion Models [63.14903268958398]
  Conditional score-based diffusion model (SBDM) is for conditional generation of target data with paired data as condition, and has achieved great success in image translation. To tackle the applications with partially paired or even unpaired dataset, we propose a novel Optimal Transport-guided Conditional Score-based diffusion model (OTCS) in this paper.
  arXiv  Detail & Related papers  (2023-11-02T13:28:44Z)
• Arbitrary Distributions Mapping via SyMOT-Flow: A Flow-based Approach Integrating Maximum Mean Discrepancy and Optimal Transport [2.7309692684728617]
  We introduce a novel model called SyMOT-Flow that trains an invertible transformation by minimizing the symmetric maximum mean discrepancy between samples from two unknown distributions. The resulting transformation leads to more stable and accurate sample generation.
  arXiv  Detail & Related papers  (2023-08-26T08:39:16Z)
• Score-based Continuous-time Discrete Diffusion Models [102.65769839899315]
  We extend diffusion models to discrete variables by introducing a Markov jump process where the reverse process denoises via a continuous-time Markov chain. We show that an unbiased estimator can be obtained via simple matching the conditional marginal distributions. We demonstrate the effectiveness of the proposed method on a set of synthetic and real-world music and image benchmarks.
  arXiv  Detail & Related papers  (2022-11-30T05:33:29Z)
• Efficient CDF Approximations for Normalizing Flows [64.60846767084877]
  We build upon the diffeomorphic properties of normalizing flows to estimate the cumulative distribution function (CDF) over a closed region. Our experiments on popular flow architectures and UCI datasets show a marked improvement in sample efficiency as compared to traditional estimators.
  arXiv  Detail & Related papers  (2022-02-23T06:11:49Z)
• Autoregressive Score Matching [113.4502004812927]
  We propose autoregressive conditional score models (AR-CSM) where we parameterize the joint distribution in terms of the derivatives of univariable log-conditionals (scores) For AR-CSM models, this divergence between data and model distributions can be computed and optimized efficiently, requiring no expensive sampling or adversarial training. We show with extensive experimental results that it can be applied to density estimation on synthetic data, image generation, image denoising, and training latent variable models with implicit encoders.
  arXiv  Detail & Related papers  (2020-10-24T07:01:24Z)
• Statistical Optimal Transport posed as Learning Kernel Embedding [0.0]
  This work takes the novel approach of posing statistical Optimal Transport (OT) as that of learning the transport plan's kernel mean embedding from sample based estimates of marginal embeddings. A key result is that, under very mild conditions, $epsilon$-optimal recovery of the transport plan as well as the Barycentric-projection based transport map is possible with a sample complexity that is completely dimension-free.
  arXiv  Detail & Related papers  (2020-02-08T14:58:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.