Injective Flows for parametric hypersurfaces
- URL: http://arxiv.org/abs/2406.09116v1
- Date: Thu, 13 Jun 2024 13:43:59 GMT
- Title: Injective Flows for parametric hypersurfaces
- Authors: Marcello Massimo Negri, Jonathan Aellen, Volker Roth,
- Abstract summary: We show that for parametric hypersurfaces we can compute the Jacobian determinant exactly and efficiently, with the same cost as NFs.
We showcase the relevance of modeling densities on hypersurfaces in two settings.
- Score: 1.4623202528810306
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Normalizing Flows (NFs) are powerful and efficient models for density estimation. When modeling densities on manifolds, NFs can be generalized to injective flows but the Jacobian determinant becomes computationally prohibitive. Current approaches either consider bounds on the log-likelihood or rely on some approximations of the Jacobian determinant. In contrast, we propose injective flows for parametric hypersurfaces and show that for such manifolds we can compute the Jacobian determinant exactly and efficiently, with the same cost as NFs. Furthermore, we show that for the subclass of star-like manifolds we can extend the proposed framework to always allow for a Cartesian representation of the density. We showcase the relevance of modeling densities on hypersurfaces in two settings. Firstly, we introduce a novel Objective Bayesian approach to penalized likelihood models by interpreting level-sets of the penalty as star-like manifolds. Secondly, we consider Bayesian mixture models and introduce a general method for variational inference by defining the posterior of mixture weights on the probability simplex.
Related papers
- Mixed Gaussian Flow for Diverse Trajectory Prediction [78.00204650749453]
We propose a flow-based model to transform a mixed Gaussian prior into the future trajectory manifold.
The model shows a better capacity for generating diverse trajectory patterns.
We also demonstrate that it can generate diverse, controllable, and out-of-distribution trajectories.
arXiv Detail & Related papers (2024-02-19T15:48:55Z) - Scaling Riemannian Diffusion Models [68.52820280448991]
We show that our method enables us to scale to high dimensional tasks on nontrivial manifold.
We model QCD densities on $SU(n)$ lattices and contrastively learned embeddings on high dimensional hyperspheres.
arXiv Detail & Related papers (2023-10-30T21:27:53Z) - Sobolev Space Regularised Pre Density Models [51.558848491038916]
We propose a new approach to non-parametric density estimation that is based on regularizing a Sobolev norm of the density.
This method is statistically consistent, and makes the inductive validation model clear and consistent.
arXiv Detail & Related papers (2023-07-25T18:47:53Z) - Differentiating Metropolis-Hastings to Optimize Intractable Densities [51.16801956665228]
We develop an algorithm for automatic differentiation of Metropolis-Hastings samplers.
We apply gradient-based optimization to objectives expressed as expectations over intractable target densities.
arXiv Detail & Related papers (2023-06-13T17:56:02Z) - Stochastic Interpolants: A Unifying Framework for Flows and Diffusions [16.95541777254722]
A class of generative models that unifies flow-based and diffusion-based methods is introduced.
These models extend the framework proposed in Albergo & VandenEijnden (2023), enabling the use of a broad class of continuous-time processes called stochastic interpolants'
These interpolants are built by combining data from the two prescribed densities with an additional latent variable that shapes the bridge in a flexible way.
arXiv Detail & Related papers (2023-03-15T17:43:42Z) - Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations
and Affine Invariance [10.892894776497165]
We study gradient flows in both probability density space and Gaussian space.
The flow in the Gaussian space may be understood as a Gaussian approximation of the flow.
arXiv Detail & Related papers (2023-02-21T21:44:08Z) - Adaptive deep density approximation for fractional Fokker-Planck
equations [6.066542157374599]
We present an explicit PDF model induced by a flow-based deep generative model, KRnet, which constructs a transport map from a simple distribution to the target distribution.
We consider two methods to approximate the fractional Laplacian.
Based on these two different ways for the approximation of the fractional Laplacian, we propose two models, MCNF and GRBFNF, to approximate stationary FPEs and time-dependent FPEs.
arXiv Detail & Related papers (2022-10-26T00:58:17Z) - Semi-Discrete Normalizing Flows through Differentiable Tessellation [31.474420819149724]
We propose a tessellation-based approach that learns quantization boundaries on a continuous space, complete with exact likelihood evaluations.
This is done through constructing normalizing flows on convex polytopes parameterized through a differentiable Voronoi tessellation.
We show improvements over existing methods across a range of structured data modalities, and find that we can achieve a significant gain from just adding Voronoi mixtures to a baseline model.
arXiv Detail & Related papers (2022-03-14T03:06:31Z) - Efficient CDF Approximations for Normalizing Flows [64.60846767084877]
We build upon the diffeomorphic properties of normalizing flows to estimate the cumulative distribution function (CDF) over a closed region.
Our experiments on popular flow architectures and UCI datasets show a marked improvement in sample efficiency as compared to traditional estimators.
arXiv Detail & Related papers (2022-02-23T06:11:49Z) - Moser Flow: Divergence-based Generative Modeling on Manifolds [49.04974733536027]
Moser Flow (MF) is a new class of generative models within the family of continuous normalizing flows (CNF)
MF does not require invoking or backpropagating through an ODE solver during training.
We demonstrate for the first time the use of flow models for sampling from general curved surfaces.
arXiv Detail & Related papers (2021-08-18T09:00:24Z) - Jacobian Determinant of Normalizing Flows [7.124391555099448]
Normalizing flows learn a diffeomorphic mapping between the target and base distribution.
Jacobian determinant of that mapping forms another real-valued function.
To stabilize normalizing flows training, it is required to maintain a balance between the expansiveness and contraction of volume.
arXiv Detail & Related papers (2021-02-12T14:09:28Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.