Flows for simultaneous manifold learning and density estimation

• URL: http://arxiv.org/abs/2003.13913v3
• Date: Fri, 13 Nov 2020 16:10:19 GMT
• Title: Flows for simultaneous manifold learning and density estimation
• Authors: Johann Brehmer and Kyle Cranmer
• Abstract summary: manifold-learning flows (M-flows) represent datasets with a manifold structure more faithfully. M-flows learn the data manifold and allow for better inference than standard flows in the ambient data space.
• Score: 12.451050883955071
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce manifold-learning flows (M-flows), a new class of generative models that simultaneously learn the data manifold as well as a tractable probability density on that manifold. Combining aspects of normalizing flows, GANs, autoencoders, and energy-based models, they have the potential to represent datasets with a manifold structure more faithfully and provide handles on dimensionality reduction, denoising, and out-of-distribution detection. We argue why such models should not be trained by maximum likelihood alone and present a new training algorithm that separates manifold and density updates. In a range of experiments we demonstrate how M-flows learn the data manifold and allow for better inference than standard flows in the ambient data space.

Related papers

• Fisher Flow Matching for Generative Modeling over Discrete Data [12.69975914345141]
  We introduce Fisher-Flow, a novel flow-matching model for discrete data. Fisher-Flow takes a manifestly geometric perspective by considering categorical distributions over discrete data. We prove that the gradient flow induced by Fisher-Flow is optimal in reducing the forward KL divergence.
  arXiv  Detail & Related papers  (2024-05-23T15:02:11Z)
• Mixed Gaussian Flow for Diverse Trajectory Prediction [78.00204650749453]
  We propose a flow-based model to transform a mixed Gaussian prior into the future trajectory manifold. The model shows a better capacity for generating diverse trajectory patterns. We also demonstrate that it can generate diverse, controllable, and out-of-distribution trajectories.
  arXiv  Detail & Related papers  (2024-02-19T15:48:55Z)
• Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models [77.83923746319498]
  We propose a framework called Diff-Instruct to instruct the training of arbitrary generative models. We show that Diff-Instruct results in state-of-the-art single-step diffusion-based models. Experiments on refining GAN models show that the Diff-Instruct can consistently improve the pre-trained generators of GAN models.
  arXiv  Detail & Related papers  (2023-05-29T04:22:57Z)
• VQ-Flows: Vector Quantized Local Normalizing Flows [2.7998963147546148]
  We introduce a novel statistical framework for learning a mixture of local normalizing flows as "chart maps" over a data manifold. Our framework augments the expressivity of recent approaches while preserving the signature property of normalizing flows, that they admit exact density evaluation.
  arXiv  Detail & Related papers  (2022-03-22T09:22:18Z)
• Nonlinear Isometric Manifold Learning for Injective Normalizing Flows [58.720142291102135]
  We use isometries to separate manifold learning and density estimation. We also employ autoencoders to design embeddings with explicit inverses that do not distort the probability distribution.
  arXiv  Detail & Related papers  (2022-03-08T08:57:43Z)
• Moser Flow: Divergence-based Generative Modeling on Manifolds [49.04974733536027]
  Moser Flow (MF) is a new class of generative models within the family of continuous normalizing flows (CNF) MF does not require invoking or backpropagating through an ODE solver during training. We demonstrate for the first time the use of flow models for sampling from general curved surfaces.
  arXiv  Detail & Related papers  (2021-08-18T09:00:24Z)
• Discrete Denoising Flows [87.44537620217673]
  We introduce a new discrete flow-based model for categorical random variables: Discrete Denoising Flows (DDFs) In contrast with other discrete flow-based models, our model can be locally trained without introducing gradient bias. We show that DDFs outperform Discrete Flows on modeling a toy example, binary MNIST and Cityscapes segmentation maps, measured in log-likelihood.
  arXiv  Detail & Related papers  (2021-07-24T14:47:22Z)
• Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flows [0.0]
  Normalizing flows provide tractable density estimation by transforming a simple base distribution into a complex target distribution. Recent attempts to remedy this have introduced geometric complications that defeat a central benefit of normalizing flows: exact density estimation. We argue that composing a standard flow with a trainable conformal embedding is the most natural way to model manifold-supported data.
  arXiv  Detail & Related papers  (2021-06-09T18:00:00Z)
• Variational Mixture of Normalizing Flows [0.0]
  Deep generative models, such as generative adversarial networks autociteGAN, variational autoencoders autocitevaepaper, and their variants, have seen wide adoption for the task of modelling complex data distributions. Normalizing flows have overcome this limitation by leveraging the change-of-suchs formula for probability density functions. The present work overcomes this by using normalizing flows as components in a mixture model and devising an end-to-end training procedure for such a model.
  arXiv  Detail & Related papers  (2020-09-01T17:20:08Z)
• Semi-Supervised Learning with Normalizing Flows [54.376602201489995]
  FlowGMM is an end-to-end approach to generative semi supervised learning with normalizing flows. We show promising results on a wide range of applications, including AG-News and Yahoo Answers text data.
  arXiv  Detail & Related papers  (2019-12-30T17:36:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.