Moser Flow: Divergence-based Generative Modeling on Manifolds

• URL: http://arxiv.org/abs/2108.08052v1
• Date: Wed, 18 Aug 2021 09:00:24 GMT
• Title: Moser Flow: Divergence-based Generative Modeling on Manifolds
• Authors: Noam Rozen, Aditya Grover, Maximilian Nickel, Yaron Lipman
• Abstract summary: Moser Flow (MF) is a new class of generative models within the family of continuous normalizing flows (CNF) MF does not require invoking or backpropagating through an ODE solver during training. We demonstrate for the first time the use of flow models for sampling from general curved surfaces.
• Score: 49.04974733536027
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We are interested in learning generative models for complex geometries described via manifolds, such as spheres, tori, and other implicit surfaces. Current extensions of existing (Euclidean) generative models are restricted to specific geometries and typically suffer from high computational costs. We introduce Moser Flow (MF), a new class of generative models within the family of continuous normalizing flows (CNF). MF also produces a CNF via a solution to the change-of-variable formula, however differently from other CNF methods, its model (learned) density is parameterized as the source (prior) density minus the divergence of a neural network (NN). The divergence is a local, linear differential operator, easy to approximate and calculate on manifolds. Therefore, unlike other CNFs, MF does not require invoking or backpropagating through an ODE solver during training. Furthermore, representing the model density explicitly as the divergence of a NN rather than as a solution of an ODE facilitates learning high fidelity densities. Theoretically, we prove that MF constitutes a universal density approximator under suitable assumptions. Empirically, we demonstrate for the first time the use of flow models for sampling from general curved surfaces and achieve significant improvements in density estimation, sample quality, and training complexity over existing CNFs on challenging synthetic geometries and real-world benchmarks from the earth and climate sciences.

Related papers

• Categorical Flow Matching on Statistical Manifolds [12.646272756981672]
  We introduce a flow-matching framework on the manifold of parameterized probability measures inspired by information geometry. We develop an efficient training and sampling algorithm that overcomes numerical stability with a diffeomorphism between manifold. We manifest that SFM can learn more complex patterns on the statistical manifold where existing models often fail due to strong prior assumptions.
  arXiv  Detail & Related papers  (2024-05-26T05:50:39Z)
• Generative Modeling on Manifolds Through Mixture of Riemannian Diffusion Processes [57.396578974401734]
  We introduce a principled framework for building a generative diffusion process on general manifold. Instead of following the denoising approach of previous diffusion models, we construct a diffusion process using a mixture of bridge processes. We develop a geometric understanding of the mixture process, deriving the drift as a weighted mean of tangent directions to the data points.
  arXiv  Detail & Related papers  (2023-10-11T06:04:40Z)
• Geometric Neural Diffusion Processes [55.891428654434634]
  We extend the framework of diffusion models to incorporate a series of geometric priors in infinite-dimension modelling. We show that with these conditions, the generative functional model admits the same symmetry.
  arXiv  Detail & Related papers  (2023-07-11T16:51:38Z)
• Generalized Neural Closure Models with Interpretability [28.269731698116257]
  We develop a novel and versatile methodology of unified neural partial delay differential equations. We augment existing/low-fidelity dynamical models directly in their partial differential equation (PDE) forms with both Markovian and non-Markovian neural network (NN) closure parameterizations. We demonstrate the new generalized neural closure models (gnCMs) framework using four sets of experiments based on advecting nonlinear waves, shocks, and ocean acidification models.
  arXiv  Detail & Related papers  (2023-01-15T21:57:43Z)
• Marginalizable Density Models [14.50261153230204]
  We present a novel deep network architecture which provides closed form expressions for the probabilities, marginals and conditionals of any subset of the variables. The model also allows for parallelized sampling with only a logarithmic dependence of the time complexity on the number of variables.
  arXiv  Detail & Related papers  (2021-06-08T23:54:48Z)
• E(n) Equivariant Normalizing Flows for Molecule Generation in 3D [87.12477361140716]
  This paper introduces a generative model equivariant to Euclidean symmetries: E(n) Equivariant Normalizing Flows (E-NFs) To the best of our knowledge, this is the first likelihood-based deep generative model that generates molecules in 3D.
  arXiv  Detail & Related papers  (2021-05-19T09:28:54Z)
• Neural Manifold Ordinary Differential Equations [46.25832801867149]
  We introduce Neural Manifold Ordinary Differential Equations, which enables the construction of Manifold Continuous Normalizing Flows (MCNFs) MCNFs require only local geometry and compute probabilities with continuous change of variables. We find that leveraging continuous manifold dynamics produces a marked improvement for both density estimation and downstream tasks.
  arXiv  Detail & Related papers  (2020-06-18T03:24:58Z)
• Flows for simultaneous manifold learning and density estimation [12.451050883955071]
  manifold-learning flows (M-flows) represent datasets with a manifold structure more faithfully. M-flows learn the data manifold and allow for better inference than standard flows in the ambient data space.
  arXiv  Detail & Related papers  (2020-03-31T02:07:48Z)
• Learning Likelihoods with Conditional Normalizing Flows [54.60456010771409]
  Conditional normalizing flows (CNFs) are efficient in sampling and inference. We present a study of CNFs where the base density to output space mapping is conditioned on an input x, to model conditional densities p(y|x)
  arXiv  Detail & Related papers  (2019-11-29T19:17:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.