Universality of parametric Coupling Flows over parametric diffeomorphisms

• URL: http://arxiv.org/abs/2202.02906v2
• Date: Tue, 8 Feb 2022 07:32:34 GMT
• Title: Universality of parametric Coupling Flows over parametric diffeomorphisms
• Authors: Junlong Lyu, Zhitang Chen, Chang Feng, Wenjing Cun, Shengyu Zhu, Yanhui Geng, Zhijie Xu, Yongwei Chen
• Abstract summary: Invertible neural networks based on Coupling Flows CFlows) have various applications such as image synthesis and data compression. We prove that CFlows can approximate any diffeomorphism in Ck-norm if its layers can approximate certain single-coordinate transforms.
• Score: 13.39432329310336
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Invertible neural networks based on Coupling Flows CFlows) have various applications such as image synthesis and data compression. The approximation universality for CFlows is of paramount importance to ensure the model expressiveness. In this paper, we prove that CFlows can approximate any diffeomorphism in C^k-norm if its layers can approximate certain single-coordinate transforms. Specifically, we derive that a composition of affine coupling layers and invertible linear transforms achieves this universality. Furthermore, in parametric cases where the diffeomorphism depends on some extra parameters, we prove the corresponding approximation theorems for our proposed parametric coupling flows named Para-CFlows. In practice, we apply Para-CFlows as a neural surrogate model in contextual Bayesian optimization tasks, to demonstrate its superiority over other neural surrogate models in terms of optimization performance.

Related papers

• Fisher Flow Matching for Generative Modeling over Discrete Data [12.69975914345141]
  We introduce Fisher-Flow, a novel flow-matching model for discrete data. Fisher-Flow takes a manifestly geometric perspective by considering categorical distributions over discrete data. We prove that the gradient flow induced by Fisher-Flow is optimal in reducing the forward KL divergence.
  arXiv  Detail & Related papers  (2024-05-23T15:02:11Z)
• The Convex Landscape of Neural Networks: Characterizing Global Optima and Stationary Points via Lasso Models [75.33431791218302]
  Deep Neural Network Network (DNN) models are used for programming purposes. In this paper we examine the use of convex neural recovery models. We show that all the stationary non-dimensional objective objective can be characterized as the standard a global subsampled convex solvers program. We also show that all the stationary non-dimensional objective objective can be characterized as the standard a global subsampled convex solvers program.
  arXiv  Detail & Related papers  (2023-12-19T23:04:56Z)
• FineMorphs: Affine-diffeomorphic sequences for regression [1.1421942894219896]
  The model states are optimally "reshaped" by diffeomorphisms generated by smooth vector fields during learning. Affine transformations and vector fields are optimized within an optimal control setting. The model can naturally reduce (or increase) dimensionality and adapt to large datasets via suboptimal vector fields.
  arXiv  Detail & Related papers  (2023-05-26T20:54:18Z)
• $\mathcal{C}^k$-continuous Spline Approximation with TensorFlow Gradient Descent Optimizers [2.0305676256390934]
  We present an "out-of-the-box" application of Machine Learning (ML)s for an industrial optimization problem. We introduce a piecewise model (spline) for fitting of $mathcalCk$-continuos functions, which can be deployed in a cam approximation setting. We then use the gradient descent optimization provided by the machine learning framework to optimize the model parameters with respect to approximation quality and $mathcalCk$-continuity.
  arXiv  Detail & Related papers  (2023-03-22T10:52:21Z)
• Dynamically-Scaled Deep Canonical Correlation Analysis [77.34726150561087]
  Canonical Correlation Analysis (CCA) is a method for feature extraction of two views by finding maximally correlated linear projections of them. We introduce a novel dynamic scaling method for training an input-dependent canonical correlation model.
  arXiv  Detail & Related papers  (2022-03-23T12:52:49Z)
• Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimization [8.683116789109462]
  This paper introduces Convex Potential Flows (CP-Flow), a natural and efficient parameterization of invertible models. CP-Flows are the gradient map of a strongly convex neural potential function. We show that CP-Flow performs competitively on standard benchmarks of density estimation and variational inference.
  arXiv  Detail & Related papers  (2020-12-10T19:36:34Z)
• Probabilistic Circuits for Variational Inference in Discrete Graphical Models [101.28528515775842]
  Inference in discrete graphical models with variational methods is difficult. Many sampling-based methods have been proposed for estimating Evidence Lower Bound (ELBO) We propose a new approach that leverages the tractability of probabilistic circuit models, such as Sum Product Networks (SPN) We show that selective-SPNs are suitable as an expressive variational distribution, and prove that when the log-density of the target model is aweighted the corresponding ELBO can be computed analytically.
  arXiv  Detail & Related papers  (2020-10-22T05:04:38Z)
• NanoFlow: Scalable Normalizing Flows with Sublinear Parameter Complexity [28.201670958962453]
  Normalizing flows (NFs) have become a prominent method for deep generative models that allow for an analytic probability density estimation and efficient synthesis. We present an alternative parameterization scheme called NanoFlow, which uses a single neural density estimator to model multiple transformation stages.
  arXiv  Detail & Related papers  (2020-06-11T09:35:00Z)
• Normalizing Flows with Multi-Scale Autoregressive Priors [131.895570212956]
  We introduce channel-wise dependencies in their latent space through multi-scale autoregressive priors (mAR) Our mAR prior for models with split coupling flow layers (mAR-SCF) can better capture dependencies in complex multimodal data. We show that mAR-SCF allows for improved image generation quality, with gains in FID and Inception scores compared to state-of-the-art flow-based models.
  arXiv  Detail & Related papers  (2020-04-08T09:07:11Z)
• Supervised Learning for Non-Sequential Data: A Canonical Polyadic Decomposition Approach [85.12934750565971]
  Efficient modelling of feature interactions underpins supervised learning for non-sequential tasks. To alleviate this issue, it has been proposed to implicitly represent the model parameters as a tensor. For enhanced expressiveness, we generalize the framework to allow feature mapping to arbitrarily high-dimensional feature vectors.
  arXiv  Detail & Related papers  (2020-01-27T22:38:40Z)
• Model Fusion via Optimal Transport [64.13185244219353]
  We present a layer-wise model fusion algorithm for neural networks. We show that this can successfully yield "one-shot" knowledge transfer between neural networks trained on heterogeneous non-i.i.d. data.
  arXiv  Detail & Related papers  (2019-10-12T22:07:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.