Joint Manifold Learning and Density Estimation Using Normalizing Flows

• URL: http://arxiv.org/abs/2206.03293v1
• Date: Tue, 7 Jun 2022 13:35:14 GMT
• Title: Joint Manifold Learning and Density Estimation Using Normalizing Flows
• Authors: Seyedeh Fatemeh Razavi, Mohammad Mahdi Mehmanchi, Reshad Hosseini, Mostafa Tavassolipour
• Abstract summary: We introduce two approaches, namely per-pixel penalized log-likelihood and hierarchical training, to answer the question. We propose a single-step method for joint manifold learning and density estimation by disentangling the transformed space. Results validate the superiority of the proposed methods in simultaneous manifold learning and density estimation.
• Score: 4.939777212813711
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Based on the manifold hypothesis, real-world data often lie on a low-dimensional manifold, while normalizing flows as a likelihood-based generative model are incapable of finding this manifold due to their structural constraints. So, one interesting question arises: $\textit{"Can we find sub-manifold(s) of data in normalizing flows and estimate the density of the data on the sub-manifold(s)?"}$. In this paper, we introduce two approaches, namely per-pixel penalized log-likelihood and hierarchical training, to answer the mentioned question. We propose a single-step method for joint manifold learning and density estimation by disentangling the transformed space obtained by normalizing flows to manifold and off-manifold parts. This is done by a per-pixel penalized likelihood function for learning a sub-manifold of the data. Normalizing flows assume the transformed data is Gaussianizationed, but this imposed assumption is not necessarily true, especially in high dimensions. To tackle this problem, a hierarchical training approach is employed to improve the density estimation on the sub-manifold. The results validate the superiority of the proposed methods in simultaneous manifold learning and density estimation using normalizing flows in terms of generated image quality and likelihood.

Related papers

• Implicit Manifold Gaussian Process Regression [49.0787777751317]
  Gaussian process regression is widely used to provide well-calibrated uncertainty estimates. It struggles with high-dimensional data because of the implicit low-dimensional manifold upon which the data actually lies. In this paper we propose a technique capable of inferring implicit structure directly from data (labeled and unlabeled) in a fully differentiable way.
  arXiv  Detail & Related papers  (2023-10-30T09:52:48Z)
• Canonical normalizing flows for manifold learning [14.377143992248222]
  We propose a canonical manifold learning flow method, where a novel objective enforces the transformation matrix to have few prominent and non-degenerate basis functions. Canonical manifold flow yields a more efficient use of the latent space, automatically generating fewer prominent and distinct dimensions to represent data.
  arXiv  Detail & Related papers  (2023-10-19T13:48:05Z)
• Out-of-distribution detection using normalizing flows on the data manifold [3.725042082196983]
  This study investigates the effect of manifold learning using normalizing flows on out-of-distribution detection. We show that manifold learning improves the out-of-distribution detection ability of a class of likelihood-based models known as normalizing flows.
  arXiv  Detail & Related papers  (2023-08-26T07:35:16Z)
• Sobolev Space Regularised Pre Density Models [51.558848491038916]
  We propose a new approach to non-parametric density estimation that is based on regularizing a Sobolev norm of the density. This method is statistically consistent, and makes the inductive validation model clear and consistent.
  arXiv  Detail & Related papers  (2023-07-25T18:47:53Z)
• Convolutional Filtering on Sampled Manifolds [122.06927400759021]
  We show that convolutional filtering on a sampled manifold converges to continuous manifold filtering. Our findings are further demonstrated empirically on a problem of navigation control.
  arXiv  Detail & Related papers  (2022-11-20T19:09:50Z)
• ManiFlow: Implicitly Representing Manifolds with Normalizing Flows [145.9820993054072]
  Normalizing Flows (NFs) are flexible explicit generative models that have been shown to accurately model complex real-world data distributions. We propose an optimization objective that recovers the most likely point on the manifold given a sample from the perturbed distribution. Finally, we focus on 3D point clouds for which we utilize the explicit nature of NFs, i.e. surface normals extracted from the gradient of the log-likelihood and the log-likelihood itself.
  arXiv  Detail & Related papers  (2022-08-18T16:07:59Z)
• Nonlinear Isometric Manifold Learning for Injective Normalizing Flows [58.720142291102135]
  We use isometries to separate manifold learning and density estimation. We also employ autoencoders to design embeddings with explicit inverses that do not distort the probability distribution.
  arXiv  Detail & Related papers  (2022-03-08T08:57:43Z)
• Resampling Base Distributions of Normalizing Flows [0.0]
  We introduce a base distribution for normalizing flows based on learned rejection sampling. We develop suitable learning algorithms using both maximizing the log-likelihood and the optimization of the reverse Kullback-Leibler divergence.
  arXiv  Detail & Related papers  (2021-10-29T14:44:44Z)
• Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flows [0.0]
  Normalizing flows provide tractable density estimation by transforming a simple base distribution into a complex target distribution. Recent attempts to remedy this have introduced geometric complications that defeat a central benefit of normalizing flows: exact density estimation. We argue that composing a standard flow with a trainable conformal embedding is the most natural way to model manifold-supported data.
  arXiv  Detail & Related papers  (2021-06-09T18:00:00Z)
• Rectangular Flows for Manifold Learning [38.63646804834534]
  Normalizing flows are invertible neural networks with tractable change-of-volume terms. Data of interest is typically assumed to live in some (often unknown) low-dimensional manifold embedded in high-dimensional ambient space. We propose two methods to tractably the gradient of this term with respect to the parameters of the model.
  arXiv  Detail & Related papers  (2021-06-02T18:30:39Z)
• Learning Likelihoods with Conditional Normalizing Flows [54.60456010771409]
  Conditional normalizing flows (CNFs) are efficient in sampling and inference. We present a study of CNFs where the base density to output space mapping is conditioned on an input x, to model conditional densities p(y|x)
  arXiv  Detail & Related papers  (2019-11-29T19:17:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.