Injective flows for star-like manifolds

• URL: http://arxiv.org/abs/2406.09116v2
• Date: Thu, 10 Oct 2024 16:09:54 GMT
• Title: Injective flows for star-like manifolds
• Authors: Marcello Massimo Negri, Jonathan Aellen, Volker Roth,
• Abstract summary: We show that we can compute the Jacobian determinant exactly and efficiently, with the same cost as NFs, for star-like manifold densities. This is particularly relevant for variational inference settings, where no samples are available and only some unnormalized target is known.
• Score: 1.4623202528810306
• License:
• Abstract: Normalizing Flows (NFs) are powerful and efficient models for density estimation. When modeling densities on manifolds, NFs can be generalized to injective flows but the Jacobian determinant becomes computationally prohibitive. Current approaches either consider bounds on the log-likelihood or rely on some approximations of the Jacobian determinant. In contrast, we propose injective flows for star-like manifolds and show that for such manifolds we can compute the Jacobian determinant exactly and efficiently, with the same cost as NFs. This aspect is particularly relevant for variational inference settings, where no samples are available and only some unnormalized target is known. Among many, we showcase the relevance of modeling densities on star-like manifolds in two settings. Firstly, we introduce a novel Objective Bayesian approach for penalized likelihood models by interpreting level-sets of the penalty as star-like manifolds. Secondly, we consider probabilistic mixing models and introduce a general method for variational inference by defining the posterior of mixture weights on the probability simplex.

Related papers

• Mixed Gaussian Flow for Diverse Trajectory Prediction [78.00204650749453]
  We propose a flow-based model to transform a mixed Gaussian prior into the future trajectory manifold. The model shows a better capacity for generating diverse trajectory patterns. We also demonstrate that it can generate diverse, controllable, and out-of-distribution trajectories.
  arXiv  Detail & Related papers  (2024-02-19T15:48:55Z)
• Quantum Normalizing Flows for Anomaly Detection [23.262276593120305]
  We introduce Normalizing Flows for Quantum architectures, describe how to model and optimize such a flow and evaluate our method on example datasets. Our proposed models show competitive performance for anomaly detection compared to classical methods. In the experiments we compare our performance to isolation forests (IF), the local outlier factor (LOF) or single-class SVMs.
  arXiv  Detail & Related papers  (2024-02-05T10:28:20Z)
• Differentiating Metropolis-Hastings to Optimize Intractable Densities [51.16801956665228]
  We develop an algorithm for automatic differentiation of Metropolis-Hastings samplers. We apply gradient-based optimization to objectives expressed as expectations over intractable target densities.
  arXiv  Detail & Related papers  (2023-06-13T17:56:02Z)
• Stochastic Interpolants: A Unifying Framework for Flows and Diffusions [16.95541777254722]
  A class of generative models that unifies flow-based and diffusion-based methods is introduced. These models extend the framework proposed in Albergo & VandenEijnden (2023), enabling the use of a broad class of continuous-time processes called stochastic interpolants' These interpolants are built by combining data from the two prescribed densities with an additional latent variable that shapes the bridge in a flexible way.
  arXiv  Detail & Related papers  (2023-03-15T17:43:42Z)
• Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariance [10.153270126742369]
  We study gradient flows in both probability density space and Gaussian space. The flow in the Gaussian space may be understood as a Gaussian approximation of the flow.
  arXiv  Detail & Related papers  (2023-02-21T21:44:08Z)
• Matching Normalizing Flows and Probability Paths on Manifolds [57.95251557443005]
  Continuous Normalizing Flows (CNFs) are generative models that transform a prior distribution to a model distribution by solving an ordinary differential equation (ODE) We propose to train CNFs by minimizing probability path divergence (PPD), a novel family of divergences between the probability density path generated by the CNF and a target probability density path. We show that CNFs learned by minimizing PPD achieve state-of-the-art results in likelihoods and sample quality on existing low-dimensional manifold benchmarks.
  arXiv  Detail & Related papers  (2022-07-11T08:50:19Z)
• Nonlinear Isometric Manifold Learning for Injective Normalizing Flows [58.720142291102135]
  We use isometries to separate manifold learning and density estimation. We also employ autoencoders to design embeddings with explicit inverses that do not distort the probability distribution.
  arXiv  Detail & Related papers  (2022-03-08T08:57:43Z)
• Efficient CDF Approximations for Normalizing Flows [64.60846767084877]
  We build upon the diffeomorphic properties of normalizing flows to estimate the cumulative distribution function (CDF) over a closed region. Our experiments on popular flow architectures and UCI datasets show a marked improvement in sample efficiency as compared to traditional estimators.
  arXiv  Detail & Related papers  (2022-02-23T06:11:49Z)
• Resampling Base Distributions of Normalizing Flows [0.0]
  We introduce a base distribution for normalizing flows based on learned rejection sampling. We develop suitable learning algorithms using both maximizing the log-likelihood and the optimization of the reverse Kullback-Leibler divergence.
  arXiv  Detail & Related papers  (2021-10-29T14:44:44Z)
• Jacobian Determinant of Normalizing Flows [7.124391555099448]
  Normalizing flows learn a diffeomorphic mapping between the target and base distribution. Jacobian determinant of that mapping forms another real-valued function. To stabilize normalizing flows training, it is required to maintain a balance between the expansiveness and contraction of volume.
  arXiv  Detail & Related papers  (2021-02-12T14:09:28Z)
• SurVAE Flows: Surjections to Bridge the Gap between VAEs and Flows [78.77808270452974]
  SurVAE Flows is a modular framework for composable transformations that encompasses VAEs and normalizing flows. We show that several recently proposed methods, including dequantization and augmented normalizing flows, can be expressed as SurVAE Flows.
  arXiv  Detail & Related papers  (2020-07-06T13:13:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.