Neural Diffeomorphic Non-uniform B-spline Flows
- URL: http://arxiv.org/abs/2304.04555v2
- Date: Tue, 11 Apr 2023 06:12:44 GMT
- Title: Neural Diffeomorphic Non-uniform B-spline Flows
- Authors: Seongmin Hong, Se Young Chun
- Abstract summary: We propose diffeomorphic non-uniform B-spline flows that are at least twice continuously differentiable while bi-Lipschitz continuous.
Our C2-diffeomorphic non-uniform B-spline flows yielded solutions better than previous spline flows and faster than smooth normalizing flows.
- Score: 19.123498909919647
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Normalizing flows have been successfully modeling a complex probability
distribution as an invertible transformation of a simple base distribution.
However, there are often applications that require more than invertibility. For
instance, the computation of energies and forces in physics requires the second
derivatives of the transformation to be well-defined and continuous. Smooth
normalizing flows employ infinitely differentiable transformation, but with the
price of slow non-analytic inverse transforms. In this work, we propose
diffeomorphic non-uniform B-spline flows that are at least twice continuously
differentiable while bi-Lipschitz continuous, enabling efficient
parametrization while retaining analytic inverse transforms based on a
sufficient condition for diffeomorphism. Firstly, we investigate the sufficient
condition for Ck-2-diffeomorphic non-uniform kth-order B-spline
transformations. Then, we derive an analytic inverse transformation of the
non-uniform cubic B-spline transformation for neural diffeomorphic non-uniform
B-spline flows. Lastly, we performed experiments on solving the force matching
problem in Boltzmann generators, demonstrating that our C2-diffeomorphic
non-uniform B-spline flows yielded solutions better than previous spline flows
and faster than smooth normalizing flows. Our source code is publicly available
at https://github.com/smhongok/Non-uniform-B-spline-Flow.
Related papers
- Delving into Discrete Normalizing Flows on SO(3) Manifold for
Probabilistic Rotation Modeling [30.09829541716024]
We propose a novel normalizing flow on SO(3) manifold.
We show that our rotation normalizing flows significantly outperform the baselines on both unconditional and conditional tasks.
arXiv Detail & Related papers (2023-04-08T06:52:02Z) - Restoration-Degradation Beyond Linear Diffusions: A Non-Asymptotic
Analysis For DDIM-Type Samplers [90.45898746733397]
We develop a framework for non-asymptotic analysis of deterministic samplers used for diffusion generative modeling.
We show that one step along the probability flow ODE can be expressed as two steps: 1) a restoration step that runs ascent on the conditional log-likelihood at some infinitesimally previous time, and 2) a degradation step that runs the forward process using noise pointing back towards the current gradient.
arXiv Detail & Related papers (2023-03-06T18:59:19Z) - Improved Convergence Rate of Stochastic Gradient Langevin Dynamics with
Variance Reduction and its Application to Optimization [50.83356836818667]
gradient Langevin Dynamics is one of the most fundamental algorithms to solve non-eps optimization problems.
In this paper, we show two variants of this kind, namely the Variance Reduced Langevin Dynamics and the Recursive Gradient Langevin Dynamics.
arXiv Detail & Related papers (2022-03-30T11:39:00Z) - Equivariant Discrete Normalizing Flows [10.867162810786361]
We focus on building equivariant normalizing flows using discrete layers.
We introduce two new equivariant flows: $G$-coupling Flows and $G$-Residual Flows.
Our construction of $G$-Residual Flows are also universal, in the sense that we prove an $G$-equivariant diffeomorphism can be exactly mapped by a $G$-residual flow.
arXiv Detail & Related papers (2021-10-16T20:16:00Z) - Smooth Normalizing Flows [0.0]
We introduce a class of smooth mixture transformations working on both compact intervals and hypertori.
We show that such inverses can be computed from forward evaluations via the inverse function theorem.
We demonstrate two advantages of such smooth flows: they allow training by force matching to simulation data and can be used as potentials in molecular dynamics simulations.
arXiv Detail & Related papers (2021-10-01T12:27:14Z) - E(n) Equivariant Normalizing Flows for Molecule Generation in 3D [87.12477361140716]
This paper introduces a generative model equivariant to Euclidean symmetries: E(n) Equivariant Normalizing Flows (E-NFs)
To the best of our knowledge, this is the first likelihood-based deep generative model that generates molecules in 3D.
arXiv Detail & Related papers (2021-05-19T09:28:54Z) - SurVAE Flows: Surjections to Bridge the Gap between VAEs and Flows [78.77808270452974]
SurVAE Flows is a modular framework for composable transformations that encompasses VAEs and normalizing flows.
We show that several recently proposed methods, including dequantization and augmented normalizing flows, can be expressed as SurVAE Flows.
arXiv Detail & Related papers (2020-07-06T13:13:22Z) - The Convolution Exponential and Generalized Sylvester Flows [82.18442368078804]
This paper introduces a new method to build linear flows, by taking the exponential of a linear transformation.
An important insight is that the exponential can be computed implicitly, which allows the use of convolutional layers.
We show that the convolution exponential outperforms other linear transformations in generative flows on CIFAR10.
arXiv Detail & Related papers (2020-06-02T19:43:36Z) - Gaussianization Flows [113.79542218282282]
We propose a new type of normalizing flow model that enables both efficient iteration of likelihoods and efficient inversion for sample generation.
Because of this guaranteed expressivity, they can capture multimodal target distributions without compromising the efficiency of sample generation.
arXiv Detail & Related papers (2020-03-04T08:15:06Z) - Fast Convergence for Langevin Diffusion with Manifold Structure [32.494158429289584]
We deal with the problem of sampling from distributions of the form p(x) propto e-beta fx) for some function f whose values and we can query.
We argue that our work is an important first step towards understanding how when there is a high degree of completion in the space of parameters the produce the same output.
arXiv Detail & Related papers (2020-02-13T15:49:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.