DFM: Interpolant-free Dual Flow Matching

• URL: http://arxiv.org/abs/2410.09246v1
• Date: Fri, 11 Oct 2024 20:46:04 GMT
• Title: DFM: Interpolant-free Dual Flow Matching
• Authors: Denis Gudovskiy, Tomoyuki Okuno, Yohei Nakata,
• Abstract summary: We propose an interpolant-free dual flow matching (DFM) approach without explicit assumptions about the modeled vector field. Experiments with the SMAP unsupervised anomaly detection show advantages of DFM when compared to the CNF trained with either maximum likelihood or FM objectives.
• Score: 0.8192907805418583
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Continuous normalizing flows (CNFs) can model data distributions with expressive infinite-length architectures. But this modeling involves computationally expensive process of solving an ordinary differential equation (ODE) during maximum likelihood training. Recently proposed flow matching (FM) framework allows to substantially simplify the training phase using a regression objective with the interpolated forward vector field. In this paper, we propose an interpolant-free dual flow matching (DFM) approach without explicit assumptions about the modeled vector field. DFM optimizes the forward and, additionally, a reverse vector field model using a novel objective that facilitates bijectivity of the forward and reverse transformations. Our experiments with the SMAP unsupervised anomaly detection show advantages of DFM when compared to the CNF trained with either maximum likelihood or FM objectives with the state-of-the-art performance metrics.

Related papers

• Local Flow Matching Generative Models [19.859984725284896]
  Flow Matching (FM) is a simulation-free method for learning a continuous and invertible flow to interpolate between two distributions. We introduce Local Flow Matching (LFM), which learns a sequence of FM sub-models and each matches a diffusion process up to the time of the step size in the data-to-noise direction. In experiments, we demonstrate the improved training efficiency and competitive generative performance of LFM compared to FM.
  arXiv  Detail & Related papers  (2024-10-03T14:53:10Z)
• Stream-level flow matching from a Bayesian decision theoretic perspective [4.935875591615496]
  Flow matching (FM) is a family of training algorithms for fitting continuous normalizing flows (CNFs) We show that viewing CFM training from a Bayesian decision theoretic perspective on parameter estimation opens the door to generalizations of CFM algorithms.
  arXiv  Detail & Related papers  (2024-09-30T15:47:22Z)
• Flow matching achieves almost minimax optimal convergence [50.38891696297888]
  Flow matching (FM) has gained significant attention as a simulation-free generative model. This paper discusses the convergence properties of FM for large sample size under the $p$-Wasserstein distance. We establish that FM can achieve an almost minimax optimal convergence rate for $1 leq p leq 2$, presenting the first theoretical evidence that FM can reach convergence rates comparable to those of diffusion models.
  arXiv  Detail & Related papers  (2024-05-31T14:54:51Z)
• Switched Flow Matching: Eliminating Singularities via Switching ODEs [12.273757042838675]
  Continuous-time generative models, such as Flow Matching (FM), construct probability paths to transport between one distribution and another. During inference, however, the learned model often requires multiple neural network evaluations to accurately integrate the flow. We propose Switched FM (SFM), that eliminates singularities via switching ODEs, as opposed to using a uniform ODE in FM.
  arXiv  Detail & Related papers  (2024-05-19T16:21:04Z)
• Optimal Flow Matching: Learning Straight Trajectories in Just One Step [89.37027530300617]
  We develop and theoretically justify the novel Optimal Flow Matching approach. It allows recovering the straight OT displacement for the quadratic transport in just one FM step. The main idea of our approach is the employment of vector field for FM which are parameterized by convex functions.
  arXiv  Detail & Related papers  (2024-03-19T19:44:54Z)
• Explicit Flow Matching: On The Theory of Flow Matching Algorithms with Applications [3.5409403011214295]
  This paper proposes a novel method, Explicit Flow Matching (ExFM), for training and analyzing flow-based generative models. ExFM leverages a theoretically grounded loss function, ExFM loss, to demonstrably reduce variance during training, leading to faster convergence and more stable learning.
  arXiv  Detail & Related papers  (2024-02-05T17:45:12Z)
• Diffusion models for probabilistic programming [56.47577824219207]
  Diffusion Model Variational Inference (DMVI) is a novel method for automated approximate inference in probabilistic programming languages (PPLs) DMVI is easy to implement, allows hassle-free inference in PPLs without the drawbacks of, e.g., variational inference using normalizing flows, and does not make any constraints on the underlying neural network model.
  arXiv  Detail & Related papers  (2023-11-01T12:17:05Z)
• Improving and generalizing flow-based generative models with minibatch optimal transport [90.01613198337833]
  We introduce the generalized conditional flow matching (CFM) technique for continuous normalizing flows (CNFs) CFM features a stable regression objective like that used to train the flow in diffusion models but enjoys the efficient inference of deterministic flow models. A variant of our objective is optimal transport CFM (OT-CFM), which creates simpler flows that are more stable to train and lead to faster inference.
  arXiv  Detail & Related papers  (2023-02-01T14:47:17Z)
• Efficient CDF Approximations for Normalizing Flows [64.60846767084877]
  We build upon the diffeomorphic properties of normalizing flows to estimate the cumulative distribution function (CDF) over a closed region. Our experiments on popular flow architectures and UCI datasets show a marked improvement in sample efficiency as compared to traditional estimators.
  arXiv  Detail & Related papers  (2022-02-23T06:11:49Z)
• Learning Likelihoods with Conditional Normalizing Flows [54.60456010771409]
  Conditional normalizing flows (CNFs) are efficient in sampling and inference. We present a study of CNFs where the base density to output space mapping is conditioned on an input x, to model conditional densities p(y|x)
  arXiv  Detail & Related papers  (2019-11-29T19:17:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.