Related papers: Learning Straight Flows by Learning Curved Interpolants

Learning Straight Flows by Learning Curved Interpolants

URL: http://arxiv.org/abs/2503.20719v1
Date: Wed, 26 Mar 2025 16:54:56 GMT
Title: Learning Straight Flows by Learning Curved Interpolants
Authors: Shiv Shankar, Tomas Geffner,
Abstract summary: Flow matching models typically use linear interpolants to define the forward/noise addition process.<n>This, together with the independent coupling between noise and target distributions, yields a vector field which is often non-straight.<n>We propose to learn flexible (potentially curved) interpolants in order to learn straight vector fields to enable faster generation.
Score: 19.42604535211923
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Flow matching models typically use linear interpolants to define the forward/noise addition process. This, together with the independent coupling between noise and target distributions, yields a vector field which is often non-straight. Such curved fields lead to a slow inference/generation process. In this work, we propose to learn flexible (potentially curved) interpolants in order to learn straight vector fields to enable faster generation. We formulate this via a multi-level optimization problem and propose an efficient approximate procedure to solve it. Our framework provides an end-to-end and simulation-free optimization procedure, which can be leveraged to learn straight line generative trajectories.

Related papers

Flow Matching: Markov Kernels, Stochastic Processes and Transport Plans [1.9766522384767222]
Flow matching techniques can be used to solve inverse problems.<n>We show how flow matching can be used for solving inverse problems.<n>We briefly address continuous normalizing flows and score matching techniques.
arXiv Detail & Related papers (2025-01-28T10:28:17Z)
Optimal Flow Matching: Learning Straight Trajectories in Just One Step [89.37027530300617]
We develop and theoretically justify the novel textbf Optimal Flow Matching (OFM) approach. It allows recovering the straight OT displacement for the quadratic transport in just one FM step. The main idea of our approach is the employment of vector field for FM which are parameterized by convex functions.
arXiv Detail & Related papers (2024-03-19T19:44:54Z)
Dirichlet Flow Matching with Applications to DNA Sequence Design [37.12809686044779]
We develop Dirichlet flow matching on the simplex based on mixtures of Dirichlet distributions as probability paths. We provide distilled Dirichlet flow matching, which enables one-step sequence generation with minimal performance hits.
arXiv Detail & Related papers (2024-02-08T17:18:01Z)
The Curse of Unrolling: Rate of Differentiating Through Optimization [35.327233435055305]
Un differentiation approximates the solution using an iterative solver and differentiates it through the computational path. We show that we can either 1) choose a large learning rate leading to a fast convergence but accept that the algorithm may have an arbitrarily long burn-in phase or 2) choose a smaller learning rate leading to an immediate but slower convergence.
arXiv Detail & Related papers (2022-09-27T09:27:29Z)
Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow [32.459587479351846]
We present rectified flow, a surprisingly simple approach to learning (neural) ordinary differential equation (ODE) models. We show that rectified flow performs superbly on image generation, image-to-image translation, and domain adaptation.
arXiv Detail & Related papers (2022-09-07T08:59:55Z)
Manifold Interpolating Optimal-Transport Flows for Trajectory Inference [64.94020639760026]
We present a method called Manifold Interpolating Optimal-Transport Flow (MIOFlow) MIOFlow learns, continuous population dynamics from static snapshot samples taken at sporadic timepoints. We evaluate our method on simulated data with bifurcations and merges, as well as scRNA-seq data from embryoid body differentiation, and acute myeloid leukemia treatment.
arXiv Detail & Related papers (2022-06-29T22:19:03Z)
Optimization on manifolds: A symplectic approach [127.54402681305629]
We propose a dissipative extension of Dirac's theory of constrained Hamiltonian systems as a general framework for solving optimization problems. Our class of (accelerated) algorithms are not only simple and efficient but also applicable to a broad range of contexts.
arXiv Detail & Related papers (2021-07-23T13:43:34Z)
Self Normalizing Flows [65.73510214694987]
We propose a flexible framework for training normalizing flows by replacing expensive terms in the gradient by learned approximate inverses at each layer. This reduces the computational complexity of each layer's exact update from $mathcalO(D3)$ to $mathcalO(D2)$. We show experimentally that such models are remarkably stable and optimize to similar data likelihood values as their exact gradient counterparts.
arXiv Detail & Related papers (2020-11-14T09:51:51Z)
Learning to Optimize Non-Rigid Tracking [54.94145312763044]
We employ learnable optimizations to improve robustness and speed up solver convergence. First, we upgrade the tracking objective by integrating an alignment data term on deep features which are learned end-to-end through CNN. Second, we bridge the gap between the preconditioning technique and learning method by introducing a ConditionNet which is trained to generate a preconditioner.
arXiv Detail & Related papers (2020-03-27T04:40:57Z)
Learning with Differentiable Perturbed Optimizers [54.351317101356614]
We propose a systematic method to transform operations into operations that are differentiable and never locally constant. Our approach relies on perturbeds, and can be used readily together with existing solvers. We show how this framework can be connected to a family of losses developed in structured prediction, and give theoretical guarantees for their use in learning tasks.
arXiv Detail & Related papers (2020-02-20T11:11:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.