Triangular Flows for Generative Modeling: Statistical Consistency, Smoothness Classes, and Fast Rates

• URL: http://arxiv.org/abs/2112.15595v1
• Date: Fri, 31 Dec 2021 18:57:37 GMT
• Title: Triangular Flows for Generative Modeling: Statistical Consistency, Smoothness Classes, and Fast Rates
• Authors: Nicholas J. Irons and Meyer Scetbon and Soumik Pal and Zaid Harchaoui
• Abstract summary: Triangular flows, also known as Kn"othe-Rosenblatt measure couplings, comprise an important building block of normalizing flow models. We present statistical guarantees and sample complexity bounds for triangular flow statistical models.
• Score: 8.029049649310211
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Triangular flows, also known as Kn\"{o}the-Rosenblatt measure couplings, comprise an important building block of normalizing flow models for generative modeling and density estimation, including popular autoregressive flow models such as real-valued non-volume preserving transformation models (Real NVP). We present statistical guarantees and sample complexity bounds for triangular flow statistical models. In particular, we establish the statistical consistency and the finite sample convergence rates of the Kullback-Leibler estimator of the Kn\"{o}the-Rosenblatt measure coupling using tools from empirical process theory. Our results highlight the anisotropic geometry of function classes at play in triangular flows, shed light on optimal coordinate ordering, and lead to statistical guarantees for Jacobian flows. We conduct numerical experiments on synthetic data to illustrate the practical implications of our theoretical findings.

Related papers

• von Mises Quasi-Processes for Bayesian Circular Regression [57.88921637944379]
  We explore a family of expressive and interpretable distributions over circle-valued random functions. The resulting probability model has connections with continuous spin models in statistical physics. For posterior inference, we introduce a new Stratonovich-like augmentation that lends itself to fast Markov Chain Monte Carlo sampling.
  arXiv  Detail & Related papers  (2024-06-19T01:57:21Z)
• Fisher Flow Matching for Generative Modeling over Discrete Data [12.69975914345141]
  We introduce Fisher-Flow, a novel flow-matching model for discrete data. Fisher-Flow takes a manifestly geometric perspective by considering categorical distributions over discrete data. We prove that the gradient flow induced by Fisher-Flow is optimal in reducing the forward KL divergence.
  arXiv  Detail & Related papers  (2024-05-23T15:02:11Z)
• Mixed Continuous and Categorical Flow Matching for 3D De Novo Molecule Generation [0.0]
  Flow matching is a recently proposed generative modeling framework that generalizes diffusion models. We extend the flow matching framework to categorical data by constructing flows that are constrained to exist on a continuous representation of categorical data known as the probability simplex. We find that, in practice, a simpler approach that makes no accommodations for the categorical nature of the data yields equivalent or superior performance.
  arXiv  Detail & Related papers  (2024-04-30T17:37:21Z)
• Unveil Conditional Diffusion Models with Classifier-free Guidance: A Sharp Statistical Theory [87.00653989457834]
  Conditional diffusion models serve as the foundation of modern image synthesis and find extensive application in fields like computational biology and reinforcement learning. Despite the empirical success, theory of conditional diffusion models is largely missing. This paper bridges the gap by presenting a sharp statistical theory of distribution estimation using conditional diffusion models.
  arXiv  Detail & Related papers  (2024-03-18T17:08:24Z)
• Applications of flow models to the generation of correlated lattice QCD ensembles [69.18453821764075]
  Machine-learned normalizing flows can be used in the context of lattice quantum field theory to generate statistically correlated ensembles of lattice gauge fields at different action parameters. This work demonstrates how these correlations can be exploited for variance reduction in the computation of observables.
  arXiv  Detail & Related papers  (2024-01-19T18:33:52Z)
• Geometric Neural Diffusion Processes [55.891428654434634]
  We extend the framework of diffusion models to incorporate a series of geometric priors in infinite-dimension modelling. We show that with these conditions, the generative functional model admits the same symmetry.
  arXiv  Detail & Related papers  (2023-07-11T16:51:38Z)
• Gauge-equivariant flow models for sampling in lattice field theories with pseudofermions [51.52945471576731]
  This work presents gauge-equivariant architectures for flow-based sampling in fermionic lattice field theories using pseudofermions as estimators for the fermionic determinant. This is the default approach in state-of-the-art lattice field theory calculations, making this development critical to the practical application of flow models to theories such as QCD.
  arXiv  Detail & Related papers  (2022-07-18T21:13:34Z)
• Efficient CDF Approximations for Normalizing Flows [64.60846767084877]
  We build upon the diffeomorphic properties of normalizing flows to estimate the cumulative distribution function (CDF) over a closed region. Our experiments on popular flow architectures and UCI datasets show a marked improvement in sample efficiency as compared to traditional estimators.
  arXiv  Detail & Related papers  (2022-02-23T06:11:49Z)
• A Numerical Proof of Shell Model Turbulence Closure [41.94295877935867]
  We present a closure, based on deep recurrent neural networks, that quantitatively reproduces, within statistical errors, Eulerian and Lagrangian structure functions and the intermittent statistics of the energy cascade. Our results encourage the development of similar approaches for 3D Navier-Stokes turbulence.
  arXiv  Detail & Related papers  (2022-02-18T16:31:57Z)
• Latent Space Model for Higher-order Networks and Generalized Tensor Decomposition [18.07071669486882]
  We introduce a unified framework, formulated as general latent space models, to study complex higher-order network interactions. We formulate the relationship between the latent positions and the observed data via a generalized multilinear kernel as the link function. We demonstrate the effectiveness of our method on synthetic data.
  arXiv  Detail & Related papers  (2021-06-30T13:11:17Z)
• Projected Statistical Methods for Distributional Data on the Real Line with the Wasserstein Metric [0.0]
  We present a novel class of projected methods, to perform statistical analysis on a data set of probability distributions on the real line. We focus in particular on Principal Component Analysis (PCA) and regression. Several theoretical properties of the models are investigated and consistency is proven.
  arXiv  Detail & Related papers  (2021-01-22T10:24:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.