Funnels: Exact maximum likelihood with dimensionality reduction

• URL: http://arxiv.org/abs/2112.08069v1
• Date: Wed, 15 Dec 2021 12:20:25 GMT
• Title: Funnels: Exact maximum likelihood with dimensionality reduction
• Authors: Samuel Klein, John A. Raine, Sebastian Pina-Otey, Slava Voloshynovskiy, Tobias Golling
• Abstract summary: We use the SurVAE framework to construct dimension reducing surjective flows via a new layer, known as the funnel. We demonstrate its efficacy on a variety of datasets, and show it improves upon or matches the performance of existing flows while having a reduced latent space size.
• Score: 6.201770337181472
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Normalizing flows are diffeomorphic, typically dimension-preserving, models trained using the likelihood of the model. We use the SurVAE framework to construct dimension reducing surjective flows via a new layer, known as the funnel. We demonstrate its efficacy on a variety of datasets, and show it improves upon or matches the performance of existing flows while having a reduced latent space size. The funnel layer can be constructed from a wide range of transformations including restricted convolution and feed forward layers.

Related papers

• FlowIE: Efficient Image Enhancement via Rectified Flow [71.6345505427213]
  FlowIE is a flow-based framework that estimates straight-line paths from an elementary distribution to high-quality images. Our contributions are rigorously validated through comprehensive experiments on synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2024-06-01T17:29:29Z)
• Scaling Riemannian Diffusion Models [68.52820280448991]
  We show that our method enables us to scale to high dimensional tasks on nontrivial manifold. We model QCD densities on $SU(n)$ lattices and contrastively learned embeddings on high dimensional hyperspheres.
  arXiv  Detail & Related papers  (2023-10-30T21:27:53Z)
• Canonical normalizing flows for manifold learning [14.377143992248222]
  We propose a canonical manifold learning flow method, where a novel objective enforces the transformation matrix to have few prominent and non-degenerate basis functions. Canonical manifold flow yields a more efficient use of the latent space, automatically generating fewer prominent and distinct dimensions to represent data.
  arXiv  Detail & Related papers  (2023-10-19T13:48:05Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• ButterflyFlow: Building Invertible Layers with Butterfly Matrices [80.83142511616262]
  We propose a new family of invertible linear layers based on butterfly layers. Based on our invertible butterfly layers, we construct a new class of normalizing flow models called ButterflyFlow.
  arXiv  Detail & Related papers  (2022-09-28T01:58:18Z)
• General Invertible Transformations for Flow-based Generative Modeling [11.04121146441257]
  We show that many well-known invertible tranformations in reversible logic and reversible neural networks could be derived from our proposition. Next, we propose two new coupling layers that are important building blocks of flow-based generative models.
  arXiv  Detail & Related papers  (2020-11-30T17:54:43Z)
• Multi-fidelity data fusion for the approximation of scalar functions with low intrinsic dimensionality through active subspaces [0.0]
  We present a multi-fidelity approach involving active subspaces and we test it on two different high-dimensional benchmarks. In this work we present a multi-fidelity approach involving active subspaces and we test it on two different high-dimensional benchmarks.
  arXiv  Detail & Related papers  (2020-10-16T12:35:49Z)
• SurVAE Flows: Surjections to Bridge the Gap between VAEs and Flows [78.77808270452974]
  SurVAE Flows is a modular framework for composable transformations that encompasses VAEs and normalizing flows. We show that several recently proposed methods, including dequantization and augmented normalizing flows, can be expressed as SurVAE Flows.
  arXiv  Detail & Related papers  (2020-07-06T13:13:22Z)
• Normalizing Flows Across Dimensions [10.21537170623373]
  We introduce noisy injective flows (NIF), a generalization of normalizing flows that can go across dimensions. NIF explicitly map the latent space to a learnable manifold in a high-dimensional data space using injective transformations. Empirically, we demonstrate that a simple application of our method to existing flow architectures can significantly improve sample quality and yield separable data embeddings.
  arXiv  Detail & Related papers  (2020-06-23T14:47:18Z)
• Normalizing Flows with Multi-Scale Autoregressive Priors [131.895570212956]
  We introduce channel-wise dependencies in their latent space through multi-scale autoregressive priors (mAR) Our mAR prior for models with split coupling flow layers (mAR-SCF) can better capture dependencies in complex multimodal data. We show that mAR-SCF allows for improved image generation quality, with gains in FID and Inception scores compared to state-of-the-art flow-based models.
  arXiv  Detail & Related papers  (2020-04-08T09:07:11Z)
• Latent Variable Modelling with Hyperbolic Normalizing Flows [35.1659722563025]
  We introduce a novel normalizing flow over hyperbolic VAEs and Euclidean normalizing flows. Our approach achieves improved performance on density estimation, as well as reconstruction of real-world graph data.
  arXiv  Detail & Related papers  (2020-02-15T07:44:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.